How will generative AI change knowledge work?
Some frameworks to understand how it might affect your job or company
There have long been concerns that “robots will take our jobs,” usually focused on industries like manufacturing that require a lot of manual labor and thus seemed like the easiest targets for automation. Now generative AI has the potential to significantly disrupt knowledge-based jobs, as one of the most powerful aspects of the technology is its ability to read and summarize massive amounts of data using natural language, and generate text that looks like what a human would write. Below, we offer a framework for how to think about the impact generative AI could have on your work or your business, informed by conversations with several founders, engineers, researchers, and VCs working in the area.
At a high level, there are two ways that generative AI will likely impact work:
Replacement of “rote” work that AI can do faster or more cheaply than humans
Augmentation of human capabilities, where a human and AI combined can achieve more than a human or a computer alone
Let’s dive into each.
Replacement of rote work
If a job requires reading a lot of text or data and summarizing or analyzing it, with relatively little subjective judgment, it likely could be replaced by generative AI.
For example, companies like Jasper are demonstrating that a lot of copywriting, like marketing emails, blog posts, and social media content can be automated. Similarly, a lot of routine queries answered by customer support staff could probably be automated.
You can also think of generative AI cutting out parts of a knowledge worker’s job that are inefficient and repetitive. For example, AI has been shown to be more efficient and accurate than humans at data entry tasks. The process of e-discovery in law, which involves combing through thousands of emails and other business communications for evidence to use in a legal proceeding, could also be a candidate for automation. Today, e-discovery can take dozens to hundreds of hours for law firm associates and, at billing rates that often top $250/hour, is very expensive. Imagine if you could just type into an interface, “find any evidence that [X company] was trying to stifle competition” and have it return only the documents a lawyer needs to look at more closely. Or imagine that a businessperson conducting due diligence on an acquisition target could just type “give me a list of all the possible irregularities in the financial statements” and get a list of items to look into. These tasks, which are often assigned to the most junior employees in professional services organizations, could in the future be handled by AI.
Augmentation of capabilities
Generative AI also has the ability to augment human capabilities. Besides the obvious productivity increase that comes from automating work that used to be time-consuming, as illustrated above, AI could enable tasks that are nearly impossible for humans today.
For example, suppose a data scientist ramping up to a new role at a large company is trying to write a query in SQL to conduct an analysis. They will likely face many questions about which data tables to use, what the column names in the data tables mean, what caveats exist about how to interpret the data, and so on, which today they would answer by asking a coworker or tracking down the data engineer who created the table. Generative AI could enable the data scientist not only to get their basic questions answered without asking anyone (”replacing” the role of a coworker during onboarding), but also to interrogate how the data is populated several levels up in the stack. They might uncover a mistake in data aggregation that occurred several steps before the pipeline that generated the table they are trying to use—something that might never be discovered otherwise. The same way that generative AI can be used to scan a large corpus of text, you could imagine it scanning and analyzing a company’s entire codebase to help uncover unexpected errors or dependencies that are very difficult for individual engineers to catch.
As another example, generative AI is starting to be used in the medical field to accelerate the process of new drug discovery. Scientists at Absci trained a large language model which they can prompt to generate an antibody for a specific protein, allowing them to come up with candidate drugs to test vastly more quickly than they could before, on the order of 2.8 million AI-generated antibody drug candidates per week!
In both these examples, it’s important to note that AI can’t replace humans. A data scientist must still decide what questions they want to answer, how to interpret the results, and what actions should be taken in their company based on what they find, even if AI helps them execute their query correctly. And of course, after generating candidate drugs to test, scientists must still go through a lengthly process of medical trials to determine if any of those drugs are effective and safe.
Seeing how AI changed jobs in the past also provides a useful comparison point. In 2016, computer scientist Geoffrey Hinton, considered one of the “godfathers of AI,” proclaimed that computer vision advancements would render radiologists obsolete within five years, but that didn’t happen. Instead, AI has changed how radiologists work by detecting patterns that humans can’t easily see in image data and automating routine tasks, while humans still play an important role interpreting the results and making a final diagnosis. We can expect generative AI to similarly augment human performance across fields. In many cases, what a human and computer together can achieve will be more valuable than what a humans or computer can do alone.
Limitations
How far are we from this reality? The answer varies by sector, but there are still several limitations. Most notably:
Data availability: Foundation models such as OpenAI’s GPT-3/GPT-4 are trained on the corpus of available data on the internet. However, a lot of valuable domain expertise is not publicly available. To be useful for use cases like legal discovery or understanding a company’s codebase, models would need to be fine-tuned to have the right domain-specific or company-specific knowledge, which can be very expensive. (The model for drug discovery mentioned above required Absci to build its own in-house supercomputer in partnership with NVIDIA.)
Hallucinations: Generative AI models are trained to generate output text by predicting the next-most-likely word to occur in a sequence. This means they are not trained to be accurate; they can make up facts or generate code that looks plausible but doesn’t work, a phenomenon known as a “hallucination.” At least for now, generative AI still requires a human “editor” with enough expertise to determine whether the output is valid. The verification required by a human can still be very time-consuming or resource-intensive, as in the case of drug discovery cited above.
Cost: Generative AI models are very expensive to train and run, and currently don’t have access to realtime information. As companies’ corpus of knowledge is constantly expanding, if a model requires constant re-training to be useful, it may become prohibitively expensive.
We can see how big these constraints loom by considering the legal field. Even though on the face of it, a lot of legal work seems like it could be automated, the average lawyer writes very differently from the average person on the internet. A base large language model would need to be fine-tuned to be able to “think” and “speak” like a lawyer. Even then, each law firm has a distinctive style; if an associate were to use generative AI to write a memo, they would want it to be able to write in their firm’s voice. So a firm might want to fine-tune the AI based on their own data. But even that is very challenging, because the same law firm may work for clients on opposite sides of a dispute—with firewalls internally to avoid conflicts—and would not want to have a model that generates outputs that could violate confidentiality.
Additionally, it is so important for law firms to be 100% accurate that multiple lawyers told us they’d be skeptical of having AI write even the first draft of a brief, since an inaccuracy introduced by the model that doesn’t get checked properly could create very high liability and reputation damage. Legal arguments need to be bulletproof, and any inaccuracy, logical inconsistency, or omission can be exploited. Today’s large language models have clear deficiencies in this respect. Even the e-discovery use case gets complicated, because lawyers who go through documents during discovery today have to pay attention not to include information that is “privileged” (e.g. marked as A/C priv or privileged for some other reason), which a generative AI model would not be trained to detect.
To use it or not to use it?
We’ve established that generative AI has a lot of promise, but also a lot of constraints to truly be usable. To determine how it is likely to be used in your industry (assuming you don’t plan to build it in-house), ask yourself the following questions:
What are tasks that are very costly because they are in the hands of experts right now, but are not big differentiators for your business? Those tasks become good candidates for generative AI.
For what tasks is it better for you to have something fast and cheap rather than higher quality? Those tasks are also good candidates for generative AI.
How important is it to get the output 100% right vs. 50% or 80% right? If you can tolerate lower accuracy, or treat the output just as a starting point for human refinement, you might want to use generative AI.
Generative AI tools are likely to be especially useful for startups or small businesses by lowering the costs to parts of doing business that are not “core” to success. For example, some founders are using image generation tools like DALL-E or Midjourney to create the first version of their logo, which gives them a good enough result to get by until they have more reasons to hire a professional designer. A founder that is using a no-code app builder or a dev shop to get an early version of their site up and running would likely be excited to use a tool like Galileo AI to generate an early mobile app design, while a founder who considers design a core part of their value proposition and is trying to build a novel interaction pattern would not.
As another example, many founders would be happy to use Jasper to generate early marketing campaigns on social media, because their copywriting isn’t a “make or break” factor for whether their startup will succeed. By contrast, a journalist at a top publication won’t find it useful to have ChatGPT write a news article for them because they care about the voice of their writing; writing is a differentiator. (Because it is trained on all human writing on the internet, AI-generated writing reverts to the mean: it is grammatically correct, but has less “burstiness” and randomness than human-generated writing.)
Who benefits?
For the foreseeable future, humans will still need to play the role of “editor” when using generative AI. They also need to know what prompts to give the model in order for it to produce the intended output — something known as “prompt engineering” — and be able to evaluate whether the output is valid.
This implies that in the near-term, the benefit of this technology is most likely to accrue to existing experts. Senior engineers love using GitHub Copilot to get coding suggestions in real time, because 80% of the time they can accept its suggestions, and 20% of the time, if it produces something off-base, they can just ignore it and move on. By contrast, someone who doesn’t know how to code would have difficulty determining when to trust the output. Even if the code compiles correctly, it could have underlying security vulnerabilities that make it dangerous to use in production.
Who benefits and who is threatened by the application of generative AI in a given industry will also determine its adoption. If the use of generative AI threatens the work of people who are also the gatekeepers to its adoption, such as partners at law firms, it is likely to be adopted more slowly, perhaps only when there is strong pressure from clients for lower-cost versions of services enabled by automation. By contrast, if the gatekeepers stand to benefit, such as a founder deciding whether to use generative AI to make a small team more productive, or technical teams who can use tools like Copilot to increase productivity, then it is likely to take off more. Whether the technology will ultimately have more of a “democratizing” impact — enabling people to do much more than they could previously, without needing to rely on experts — or a “monopolizing” impact — giving more power to existing experts who become increasingly productive — remains to be seen.
How are you seeing generative AI used or not used in your field? Leave us a comment below!
Special thanks to Brian Tuan and Amanda Kelly for comments on an earlier draft of this post.