If you've been using OpenAI's GPT-4, you might have recently noticed that the once-heralded AI has become "lazier" and "dumber". The once-crisp responses now often suffer from weakened logic, and the model seems to struggle with even simple instructions. What has changed with GPT-4 and is there a reason behind this decline in performance?
GPT-4's Performance Slip: From Ferrari to Old Pickup
GPT-4, once lauded for its remarkable accuracy and comprehension of complex tasks, now seems less powerful. As industry insiders discuss a possible major redesign of the AI system, the AI community is left scratching their heads over the noticeable performance degradation. Developers and tech industry insiders who previously used GPT-4 to enhance their productivity, now express their frustration over its decline in efficiency and effectiveness. The major complaints revolve around the system's lessened logic and increased error rates. Issues such as forgetting to add brackets in basic software code, losing track of given information, and difficulties in following instructions are rampant. But why the sudden shift in performance?
A Radical Redesign: From Slow and Expensive to Fast and Inaccurate
Earlier this year, OpenAI was impressing the world with the launch of GPT-4. This was a step forward from its precursors, GPT-3 and GPT-3.5, and was deemed the most powerful AI model available broadly. The multimodal AI model could understand both images and text inputs, providing near-human levels of response accuracy.
However, the initial enthusiasm was soon replaced with shock as users received their bills for using GPT-4. The model was slow but very accurate and was also quite expensive to run. This was the situation until a few weeks ago when GPT-4 started operating at a quicker pace, but the performance dropped notably. This sparked discussions about a potential major change in the AI's architecture.
The Mixture of Experts (MOE) Approach
According to Sharon Zhou, CEO of Lamini, a startup that helps developers build custom large language models, it seems that OpenAI has started creating several smaller GPT-4 models that perform similarly to the large model but are less expensive to operate. This approach is called a Mixture of Experts, or MOE.
The idea behind the MOE approach is to have different models, each trained on specific tasks and subject areas. When a GPT-4 user asks a question, the new system identifies which expert model to send the query to, potentially using multiple models and combining the results.
Zhou likens this situation to the "Ship of Theseus" thought experiment, which questions whether an object that has had all its components replaced remains fundamentally the same object. "OpenAI is taking GPT-4 and turning it into a fleet of smaller ships," she said.
The Future of GPT-4: A Work In Progress?
Despite the perceived performance dip, it's crucial to understand that AI development is not a linear process. The Mixture of Experts approach could be OpenAI's attempt to balance cost and quality. And while it may seem like a step back, the smaller, specialized models might just need time to adjust and learn.
OpenAI has previously mentioned the MOE approach, noting that it enables many more parameters without increased computation cost. However, the current state of GPT-4 could likely be attributed to this training and rollout of the fleet of smaller expert GPT-4 models.
In conclusion, while it might seem like GPT-4 has grown "lazier" or "dumber," this could just be a hiccup in the journey towards an even more powerful, efficient, and cost-effective AI model. As users continue to engage with the new system, the data collected could help refine these smaller models and potentially lead to an AI system that is not only more accurate but also more economical to run. It's a balancing act, and only time will tell if OpenAI has made the right decision.