This may be almost a non-article, but the buzz surrounding ChatGPT, especially after the announcement of AI-powered Microsoft Bing, makes it necessary for every self-respecting professional to try AI tools.
For those that live under a rock and avoid all the news (I deeply respect that lifestyle choice in this day and age), ChatGPT is a language model developed by OpenAI that can chat with users and generate human-like responses to text prompts. It is trained on vast amounts of data and can understand and interpret natural language.
I’ve used good old GPT-3 professionally before, mostly to help me compose take-home assignments, but since I don’t really work with code that much, I wasn’t too sure if ChatGPT would be such a game-changer for my workflow.
Product managers, especially in startups, have to deal with a ton of different tasks on a daily basis: doing market research, generating new product ideas, thinking up new ways to gather customer feedback and then analysing it, and so on, and so on.
Could at least some of the routine tasks be made easier with ChatGPT?
Market Research
I remember spending days looking for ways to find current market data at various start-ups. Sometimes that data was needed to confirm that the idea had some merit at all. Sometimes I needed good TAM, SAM and SOM figures. Can ChatGPT help us here?
Getting the right numbers
Let’s ask it for the TAM (Total Addressable Market) of the digital fashion market.
One of these is absolutely useless (why online clothing rental?), but the other one is pretty good.
However, since ChatGPT can’t provide you with any links, and some of this info may be coming from behind paywalls, finding the exact reports it mentions won’t always be easy.
I still think that knowing where to look for this kind of info is an incredible power to have, even if the actual data you get from ChatGPT may have its issues.
The verdict:
Great way to start your research, but a very poor data source to fully rely on.
Looking at products’ competitors
ChatGPT is definitely not all-knowing, and while it can provide info on major companies and their competitors, the results are very much hit-or-miss.
Here’s an attempt to get a list of Airtable competitors. I love Airtable and have used it for numerous projects. I am aware of the real competitors, so it would be very easy for me to judge the response.
Since ChatGPT is operating with minimal knowledge of Airtable’s functionality, it focuses on some keywords, while ignoring the gist of it. Unfortunately, simple Google searches for “Airtable competitor”, “Airtable clone” or “Airtable alternative” won’t give you any easy answers either, because you’ll be bombarded with tonnes of ads for project management tools—one of Airtable’s use cases, but definitely not the only thing it could be used for.
It’s a 0:0 in ChatGPT’s competition with traditional search engines. It’s good news that at least my job is safe here. By the way, the real competitors would be Budibase, an advanced database solution, and Baserow, an open-source Airtable clone.
The verdict:
Be very cautious with this info. It’s just as bad as blindly trusting the “I’m feeling lucky” button on Google.
Content creation
ChatGPT (even GPT-3, really) is supposed to be pretty good at writing stuff. Can it help us here?
Product specifications
Hear me out, this use case is absolutely AMAZING. While you won’t get your work done for you, ChatGPT will give you a great template to start from. It’s like doing a backlog grooming with a team member that knows absolutely nothing about anything.
The verdict:
This definitely works! Use the results as a template to help you think about the real requirements, user stories and acceptance criteria (depending on your team’s approach to writing specs).
Can you generate UI Strings?
I have no idea whether all product managers do this, but I often get involved in writing short phrases that would be easy to understand.
Even given a more detailed brief, ChatGPT fails in a way even a junior designer never would be able to.
The verdict:
You can’t replace a copywriter or even a good product manager or designer who could write well, with ChatGPT. Don’t even try, you will fail spectacularly.
Customer Feedback
Product managers often need to work with customer feedback. If there is a functioning Customer Success team, we still need to analyse the incoming messages to get the prevailing themes and see what can be improved. If there isn’t, the Product team can often end up doing at least a part of customer support work. Can ChatGPT help here?
Summarising the reviews
Let’s test it out on some app reviews for a hip social networking app all my friends use.
I only used the top 6, but you could feed in more info if you desire.
The verdict:
This is a great use case. Text summarization is a task that has already had some decent solutions, but ChatGPT does it really well, so you don’t have to rely on third-party services.
Replying to reviews
A good reply can persuade the user to improve their rating of your app. It also needs a great understanding of the context. Could ChatGPT be of use here?
Not too bad! A bit too wordy for App Store, but it’s a good neutral response. If a negative review makes you angry, ChatGPT is a good way to clear up your mind.
It’s pretty similar to the actual developer response that was posted, except that one was more laconic:
The verdict:
Probably won’t help with specific bugs, but a good way to generate unique neutral responses.
Conclusion
No matter what you read here, give ChatGPT and other AI tools, including Notion AI, a try. There are many use cases where you may find them useful, even if I didn’t.
AI text tools won’t steal your job if you are competent, the same way Trello, Jira, Notion or whatever tools you use on the daily basis never did. However, if you won’t master them to amplify your work process, you will end up in the same position as artisans did when machinery came into their respective fields of work.
Perhaps there will be a demand for artisanal organic product management in the future, but even if you plan to go down that alley it won’t hurt to learn more about your future enemy.
I would love to hear more about your experience in using ChatGPT and other AI tools in day-to-day product management tasks!