📊 NVIDIA showcasing new "interfence-only tech" @ GTC in next few days (building on Groq acqui.) The pending earthquake I predicted in 2024: inference will become key to training, and many that are investing heavily in trad GPU infra will get caught out and by this coming shift. Why? There is a shortage of fresh training data to help with scaling intelligence through pretraining. To solve this, CoT reasoning will be applied within agentic architectures to perform research that axiomatically generates discoveries and insights — with such frameworks validating empirical assumptions and reasoning to prevent the generation of training data based on hallucinations or faulty thinking. New frontier knowledge insights will be formulated for use as training data in ways that both help embed new knowledge in model weights, and increase embedded "intuition," which directly supports the notional "intelligence quotient" that can be ascribed to textual synthesis by LLMs. Inference-generated training data will prove instrumental to advancing frontier LLM capabilities. Meanwhile, its generation will involve vast amounts of inference computation, and interfence-only ASICs (application specific integrated circuits) will be required to do it competitively. The only question is when agentic frameworks and LLM capabilities will advance sufficiently to make this possible. My own experiences applying agentic frameworks at work in recent months make me think that agentic generation of training data at scale may not be that far off. This is definitely something tech firms investing heavily in do-it-all GPU chips and data centers specialized to host them should think about. Once end-users get a taste for inference on ASICs, which can be 10X faster compared to the latest GPUs, they're also not going to want to do inference on anything else, at least for chat and code generation, which are very time-sensitive tasks — for example, http://caffeine.ai plans to adopt ASICs as soon as possible. It's a shame NVIDIA bought Groq, because competition is needed to advance this sector, so all eyes on Cerebras and their dinner plate sized chips!
ViewDAO