ChatGPT plugins - bad name, significant impact, scary times
Hi,
have you heard the latest joke? OpenAI safety committee gathers before the next release:
- Will this blow up?
- It shouldn't...
For me it certainly feels like it almost blew up with the latest announcement. Let's catch up and dive into that.
Saving Endangered Languages - niche and long-term strategy
ML-driven products at Trustbit
Chat GPT Plugins - bad name, significant impact
Infrastructure news
BTW, this is Rinat Abdullin, writing about a journey in building ML products in these turbulent times. You have probably subscribed to this newsletter from my website: https://abdullin.com/. The unsubscribe button should be at the bottom of this newsletter.
Saving Endangered Languages - niche and long-term strategy
We've been making some progress for creating a smart assistant for an endangered language (Bashkir). There is already a more clear product vision, first groups of testers and a usable chatbot integration. API is finally capable of working with multiple users concurrently :)
The plans are to have a robust smart assistant with good speech and knowledge capabilities. Something you can talk to about your ancestors, native food or traditions.
We are hoping to gradually extend this project to support and help more endangered languages. Bashkir language is the first one, because this is the native language of Aigiz. However, the patterns and approaches are reusable.
If this was the English language (or a similar one), there would be no work at wall - just leverage ChatGPT and all integrations available for it. Things will work magically out of the box with a little bit of prompt engineering. For us - we have pretty much to build and train things from the scratch. It is a hard work, but it is worth it in a long term.
Next task for me: create an expert system (akin to ChatGPT plugin) that could answer pointed questions about a few cultural works of Bashkir language. This is tricky and teaches us a lot about working with embeddings, multi-lingual pipelines and problematic language domains at scale.
This is a lot of fun, helps real people and also builds a unique expertise in the field.
ML-driven products at Trustbit
At the beginning of the week we had a Learning & Sharing Webinar at Trustbit, talking about ChatGPT-4 and its future impact on the industry.
Here is the Miro board, if you are interested: https://miro.com/app/board/uXjVMcDISVI=/?share_link_id=854709561824. Webinar was recorded and is in the pipeline to be published.
This has inevitably lead to concrete product discussions, within the company and with the customers. Boy, it is easier to prototype projects than ever before.
Here are a few ideas that we could already offer to the customers (who are interested in that):
Smart search boxes on websites. Instead of going for old-school search, you could ask pointed questions about "which product would be the best fit for this problem of mine?" I did prototypes, it works amazingly well!
Taking entire product catalogues and rewriting them at scale to be more human-readable and SEO-optimised. Generating SMM content for Twitter and Facebook to announce sales and marketing campaigns. I did prototypes, it works.
Offering smart intranet search and chat bots for the companies to accelerate and assist existing workflows. Think of domain-specific Copilot.
Chat-GPT-assisted draft of modular solutions that required a dedicated designer before. This is one is a lot more tricky, since that requires simulation and constraint solving, but still LLMs could help a lot here. "Please redesign the entire thing so that I have 3 WCs in the centre of the industrial complex"
Obviously, there is a difference between between prototyping and execution. Larger the company is, more difference is there. However, at least the technical aspects are doable in various configurations (including fully self-hosted scenario, as long as you have the resources to re-train).
How do I know? Because this is easy. Digitising a part of some business workflow in a Romano-germanic language is much easier than digitising subtle cultural aspects for a niche endangered languages where you have pretty much to do everything from the scratch. Building expertise in hard areas pays off.
Chat GPT Plugins - bad name, significant impact
Let's talk about the elephant in the room - new ChatGPT plugins. This is big.
OpenAI showed how they can turn their isolated chat into a platform for interacting with the online world. This is done via plugins that anybody can develop. Here is how things work in a nutshell:
Plugin can be a HTTP API with some description and an endpoint. There is no hard schema, just text!
ChatGPT will inject plugin description in a compact form into the dialog
When user asks a question, LLM might decide to invoke the plugin, doing GET or a POST request.
Results returned by the plugin will be consumed by LLM to produce a final answer.
Previously we had only bing search with a capability to browse the web. Now, more people and companies could develop their plugins. Wolfram, Kayak, Expedia and Klarna are among the others who already did.
At the beginning of the week customers were asking us "How can we make sure that the new Bing search has good access to all information from our site?" It is the end of the week and that is not the right question to ask.
If you recall, there is a chat called WeChat - the largest messaging chat in the world with more than a billion of monthly users. It is used in China for everything - ordering food, playing games, shopping, getting taxis etc.
Chances are, Microsoft (owns 49% of OpenAI) will try to have a shot at creating and controlling such a marketplace in Western world as well. Сompanies will be running to be a part of this new marketplace.
I think, in the long term this approach will have even a stronger impact on the way business is being conducted in the real world.
Here is one example I imagine most vividly.
Imagine, Microsoft has deployed OpenAI Chat GPT internally, fine-tuning it on the history of internal communications and email conversations. Imagine that they have also deployed plugins to let the new large language model index and access more real-time information: new emails, code repositories, phone conversations and customer support tickets.
Whenever any employee of the company faces a question or a decision, he/she/they could just ask a question in a 1-to-1 chat with the "Microsoft Avatar" or "Digital business twin of the company that you can talk to". This avatar will be very understanding and kind, it will also have the combined knowledge of the company at its fingertips. It will help that employee to improve the decision by pointing out problems or highlighting potential options. Might even ask at the end how did kids do in that science fair last week.
Do you know what is the scariest part of that? I don't see a single reason why this scenario is not feasible. In fact, I'm working on one, just creating avatar of a language/culture, instead of a business entity. Infrastructure and underlying principles are still the same.
Infrastructure news
Oh, speaking of the infrastructure. NVidia has just announced new GPUs, specially designed for large language models. H100 NVL that stitches together two H100 cards and offers 94GB of GPU memory. This is just for the big companies and enterprise customers, though.
Mortals like us will have to be scrappy and inventive: explore model parallelism, crowd-source prompts for InstructGPT and fine-tune existing publicly available LLM.
Scary and crazy times, if you ask me.
How are you holding up?
With best regards,
Rinat