Artificial Intelligence, what comes next?

Non-transformer models. Catalysts for tomorrow’s innovation revolution.

Jan 14, 2025

Welcome to Silicon Sands News, read across all 50 states in the US and 111 countries.
We are excited to present our latest editions on how responsible investment shapes AI's future, emphasizing the OECD AI Principles. We're not just investing in companies. We're investing in a vision where AI technologies are developed and deployed responsibly and ethically, benefiting all of humanity.

This week, we will explore what comes next for AI after the technology under ChatGPT and similar technologies.

Let’s Dive Into It . .

iPhone 17 or Blackberry?

Imagine unveiling your game-changing iPhone 17 only to realize everyone’s still raving about a decade-old Blackberry. That’s the strange disconnect in the AI world. While colossal transformer models like GPT-4 dominate the headlines, a quieter set of AI breakthroughs redefines what’s possible.
These under-the-radar technologies, so-called non-transformer AI models, are the fundamental catalysts for tomorrow’s AI revolution—sleeker, more cost-effective, and transparent enough to win over regulators and everyday users alike.
They’re inspired by nature, human logic, and creative problem-solving—and they don’t need an army of GPUs to keep running. Instead of clinging to the idea that “bigger is always better,” these alternative AI models adapt on the fly, work smoothly in edge devices, and explain themselves in ways regulators (and your average user) can understand.

If you’ve been craving an AI approach that’s as adaptable as Hollywood’s most versatile actors or meticulously designed product, look no further. This new wave of AI might be the quiet revolution you’ve been waiting for.

TL;DR

Large transformer models get the press, but non-transformer AI rapidly emerges as a lighter, more transparent, and adaptive alternative. These methods—from Liquid Neural Networks to neuromorphic hardware—consume fewer resources, adapt in real-time, and often provide better explanations for their decisions. They are particularly suited for healthcare, finance, and legal tech that demand reliability and clarity. Investors, senior executives, and founders can tap into these architectures for a more cost-effective, eco-friendly, and versatile AI strategy, potentially gaining a competitive edge in a market increasingly focused on responsible innovation.

AI has captured the public imagination with large-scale models like GPT-4. Yet beneath the headlines, a quiet but powerful wave of alternative AI methods is gaining momentum. These non-transformer approaches are poised to transform industries that need efficient, adaptable, and transparent solutions without the exorbitant costs and resource requirements associated with massive transformer-based systems. For investors seeking new avenues in technology, this rapidly evolving ecosystem is too promising to ignore.

Book a call or session with me

Non-Transformer AI Models Bring a Refreshing Change

Most of us have seen the headlines about “bigger is better” in tech—think of smartphones with more cameras or social networks with billions of users. AI has followed a similar script, with transformer models like GPT-4 gobbling up enormous amounts of data to produce eerily human-like writing or lightning-quick image recognition. But that heavy-duty approach also means transformers require tons of electricity, specialized chips, and often round-the-clock maintenance.

Non-transformer AI takes a different path. Instead of throwing endless resources at a problem, these models focus on how intelligence can adapt to changing conditions—sometimes with minimal data—and how it can explain its reasoning. Researchers might mimic how neurons fire in the human brain or infuse traditional machine learning with logical rules that make decisions more transparent. By prioritizing adaptability and clarity, these new techniques are carving out niches where trust, compliance, or real-time choices matter more than raw computing power.

Why These Alternatives Are Taking Off
Transformers aren’t going anywhere, but they can be overkill for tasks that don’t require super-high accuracy on massive data sets. Non-transformer models are often:

Less Resource-Heavy
They need less data to learn effectively and can run on smaller, cheaper hardware.
Real-Time Friendly
Many adapt their inner workings “on the fly,” making them ideal for applications such as traffic control, security monitoring, or trading systems that require instant updates.
More Transparent
Some blend data-driven insights with explicit rules, making them easier to audit—which is vital in regulated sectors like banking or healthcare.

This focus on lightweight operation and quick adaptation resonates with organizations under pressure to manage costs, meet stricter regulations, and prove their AI isn’t a black box making untraceable decisions.

How They Differ From ChatGPT

Appreciating what makes non-transformer AI so intriguing helps one understand what transformers do well and where they fall short.

Transformers excel at analyzing large amounts of information simultaneously, “paying attention” to different parts of text or images simultaneously. This multitasking approach yields incredibly sophisticated results but also demands substantial energy and computing power. It can also be tricky to explain how a transformer came to a particular conclusion or recommendation.

Non-transformer architectures zero in on efficiency and clarity. Some imitate how the human brain fires neurons only when necessary, drastically cutting energy usage. Others track how decisions evolve step by step, providing an easy-to-follow logic trail. These alternative models may not always match transformer-level performance in tasks like writing Shakespearean sonnets on demand. Still, they shine when you need nimble decision-making, robust handling of streaming data, or accountability for each outcome.

Why Pay Attention?

Non-transformer models represent the frontier in AI development, offering opportunities that align with market and regulatory trends. As corporations worldwide try to minimize carbon footprints, models that run efficiently on smaller hardware hold unique appeal. Investors can seek out startups developing neuromorphic chips or specialized AI engines for edge devices, tapping into a growing demand for on-device intelligence. Major enterprises eager to maintain a competitive edge are exploring these technologies to tackle issues like supply chain management, real-time personalization, and legal compliance. Those who support this new wave of AI will position themselves as leaders in an ecosystem likely to attract substantial investment and partnership opportunities in the coming years.

Adaptive Prowess

Liquid Neural Networks, sometimes called LNNs, stand out because they do not fix their internal parameters after training. Instead, they continue to evolve in response to incoming data. This is particularly useful when data flows continuously, such as in live monitoring systems or financial trading. Liquid networks can adjust their decision-making to reflect the latest information, reducing the risk of model “drift” in changing environments. Researchers at MIT have demonstrated that these networks can remain robust under noisy conditions, making them well-suited for healthcare, industrial IoT, and robotics tasks.

From an investment perspective, LNNs offer a double advantage–they are comparatively lightweight in energy usage and can adapt without full retraining on gigantic data sets. This combination helps reduce operational expenses and makes them more accessible to smaller organizations. In practical applications, a liquid network might power a drone that navigates unpredictable outdoor conditions, adjusting its flight patterns based on wind or weather changes. In finance, it might help a trading platform adapt to sudden market shifts more gracefully than a model fixed in its parameters. Although there is ongoing research to ensure Liquid Neural Networks remain explainable as their parameters shift, these architectures are already demonstrating real-world promise.

Spiking Neural Networks

Neuromorphic computing aims to replicate the way biological brains process information. It uses specialized hardware chips to communicate through electrical spikes rather than continuous signals. This approach allows the system to “rest” when no new events occur, leading to considerable energy savings over traditional CPU—or GPU-based solutions. Early neuromorphic chips like IBM’s TrueNorth and Intel’s Loihi have shown that tasks like visual recognition and speech processing can be accomplished with far fewer computational resources.

The adoption of neuromorphic computing could reshape industries that rely on battery-powered devices or remote sensors. For example, a wearable health monitor with a neuromorphic processor could analyze biometric data locally, alerting users to potential health issues in real time without draining the device’s battery. A swarm of small robots could navigate disaster zones more effectively if each unit can make quick, efficient decisions without constantly sending data to the cloud. Speed and energy efficiency benefits align well with the global shift toward more sustainable and decentralized technologies.

Although neuromorphic systems can be challenging to interpret due to their spiking mechanisms, some developers are adding rule-based overlays that track how each “neuron” or cluster of neurons responds to input. This approach makes it easier to audit the decision process, which becomes especially important in regulated fields or mission-critical operations. The neuromorphic sector is particularly appealing to investors because hardware ventures specializing in this area could become key suppliers for advanced robotics, Internet of Things devices, and medical implants.

Transparent Reasoning

Neurosymbolic AI tries to solve one of the biggest headaches in machine learning: explaining how a model concluded. Traditional neural networks often appear to be “black boxes,” providing answers without much insight into their reasoning. Symbolic systems, on the other hand, excel at logic and clear decision rules but struggle with unstructured data like images and natural language. Neurosymbolic AI merges these two approaches to allow models to learn directly from raw data while providing clear, rule-based rationales.

This hybrid architecture is particularly valuable in fields with regulatory or ethical constraints. A neurosymbolic system used in banking might identify suspicious activities in customer transactions and trace its conclusion to specific risk-scoring rules, providing a transparent audit trail for regulators. In drug discovery, it could propose new molecular structures that appear promising based on data patterns but also cite the chemical principles it relied on, making the results more convincing to researchers. This clarity can reduce compliance risks and build trust, which is increasingly vital for AI deployments in fields that affect public health, finance, or personal privacy.

Companies that master neurosymbolic techniques may have a significant advantage in winning contracts, especially in government or enterprise sectors where accountability is key. This approach can also help expedite processes like contract review and legal analytics by identifying relevant clauses or potential red flags and explaining precisely why those flags were raised. For investors, backing a firm working on neurosymbolic AI could mean early entry into a market that grows steadily, driven by consumer and institutional demands for interpretable, evidence-based AI decisions.

Adaptable Intelligence

Meta-learning, often described as “learning to learn,” allows AI systems to generalize across tasks rather than solving a single narrow problem. A meta-learning system is trained on many different functions until it understands the fundamentals of adaptation. When exposed to a new problem, it can pivot quickly with only a tiny amount of fresh data. This approach contrasts with conventional machine learning, which typically requires large data sets and extensive retraining whenever the task changes.

In healthcare, meta-learning can be particularly powerful. Instead of developing a new model each time researchers want to detect a specific rare disease, they can build a meta-learning engine that adapts to such scenarios after studying more common ailments. This adaptability reduces time and resource requirements, critical in life-saving diagnostics. Personalized education could also benefit, as meta-learning systems offer each student a customized learning path after a brief assessment, continually fine-tuning material based on individual progress.

Meta-learning systems, while adapting faster, need careful oversight to ensure they do not pick up biases or make leaps that defy logic. Researchers often keep detailed records of how the meta-learner updates its parameters so they can identify potential errors or biases before deploying the system widely. For investors, the appeal lies in supporting AI solutions that dramatically reduce data requirements and retraining costs. Emerging areas like robotics, real-time recommendations, or specialized medical applications could see rapid gains if they adopt meta-learning early.

Efficiency Innovations

State space models and HyperMixers are another family of architectures that tackle time series or sequence data without resorting to the transformer’s heavy attention mechanism. By maintaining or updating hidden states over time, state space models capture long-term dependencies in data with lower computational overhead. HyperMixers, which rely on streamlined multi-layer perceptrons, also reduce complexity and can be more straightforward to interpret than large attention-based systems.

These approaches are finding niche applications where real-time analysis of sequences is vital. Financial firms turn to state space models to track unfolding market trends and volatility, making quicker decisions than a large transformer allows. Creative industries can use HyperMixers for tasks like on-the-fly music generation or live video editing, integrating them into consumer-grade hardware without sacrificing performance or causing latency issues. Their less complicated internal design can also make explaining how a model arrived at a particular conclusion more approachable.

The focus on computational efficiency aligns well with the broader push for green AI. As more organizations commit to sustainability goals, adopting models with smaller carbon footprints is no longer just a cost-saving measure but also a reputational advantage. For companies and investors, exploring these architectures now can provide an edge when customers or regulators begin asking for data on energy consumption in AI services.

Ethics & Safety

Non-transformer AI models are not immune to the ethical and safety challenges of advanced machine learning. Liquid Neural Networks, which adapt their parameters in real-time, could be hijacked by adversarial attacks if they are not rigorously tested. Neuromorphic systems might behave unpredictably if their spiking neurons receive malformed inputs. Meta-learning could amplify unwanted biases if the initial tasks it studied were unrepresentative or skewed.

These challenges can be addressed with careful research, extensive simulation, and real-world pilot programs. Regulators, especially in Europe and parts of Asia, are drafting rules that may demand new certification or auditing for adaptive AI systems. Engineers working on these models must prepare documentation, plan for worst-case scenarios, and incorporate fail-safes that shut down or revert the system when anomalies arise. Although compliance can be demanding, it also allows conscientious developers and investors to set themselves apart in a market that increasingly values responsible innovation.

Transparency & Explainability

Transparency must begin at the design phase and continue through deployment. Some teams integrate data pipelines that validate inputs before reaching the AI model. Others build interpretability tools that give engineers, regulators, or end users a way to visualize how the model processes data. For instance, a neurosymbolic system might display logical pathways that explain how a conclusion was drawn. At the same time, a Liquid Neural Network may keep a log of weight updates as it encounters new information.

Strong governance policies should complement technical transparency. This can include routine bias checks, continuous system performance monitoring, and a clear channel for reporting suspected problems. Companies that embrace these practices from the start are more likely to pass regulatory scrutiny and build lasting relationships with partners who value responsible innovation. They also reduce their exposure to reputational harm if an unexpected problem arises.

From an investment standpoint, supporting transparency-focused startups and encouraging these practices in established tech firms can yield substantial reputational benefits. As regulators and the public become more aware of AI’s potential pitfalls, transparent and explainable systems will likely gain a competitive edge.

Opportunities & Potential Pitfalls

Even though the future for non-transformer AI looks bright, it is worth acknowledging that some of these technologies remain in the early stages of adoption. Neuromorphic hardware may still be expensive or specialized, limiting its short-term availability. Liquid Neural Networks can handle continuous data streams well but may require new approaches to debugging if they adapt unexpectedly. Neurosymbolic frameworks often need careful tuning to ensure that the symbolic and neural components do not conflict or produce contradictory interpretations.

Executives and investors should develop a clear strategy for approaching these challenges. They can partner with universities and research institutions for proof-of-concept studies, sponsor pilot deployments that verify performance under real-world conditions, and actively shape conversations about ethical guidelines and regulations. These steps can help reduce risks and foster a healthy ecosystem where these technologies can thrive.

Responsible AI Innovation

The world of AI is far more diverse than the headlines about massive transformer models might suggest. By looking beyond these data-hungry giants, it is possible to glimpse a future in which smaller, more adaptive, and more transparent models flourish. Liquid Neural Networks offer on-the-fly adaptation for real-time decision-making. Neuromorphic computing brings biological efficiency to devices that must conserve power. Neurosymbolic frameworks show that AI can be both powerful and logically interpretable. Meta-learning accelerates the ability to pivot to new tasks, while state space models and HyperMixers reduce computational overhead without sacrificing performance.

The payoff for those willing to invest time, money, and research into these emerging approaches could be substantial. Non-transformer AI addresses pressing needs in healthcare, finance, robotics, education, and other sectors that demand trustworthy, energy-efficient technology. Organizations embracing these models can differentiate themselves through more explicit, transparent operations while potentially reducing costs and environmental impacts.

As the conversation around AI ethics and governance evolves, these innovations promise a more balanced relationship between raw computing power and human oversight. At Silicon Sands News, we see immense potential in the non-transformer AI movement, especially for those with the foresight to participate at this early stage. By championing new ideas and holding them to high standards of explainability and responsibility, investors, founders, and executives can drive AI in a direction that benefits society as it is to the bottom line.

We look forward to watching this story unfold and hope you will join us on this journey toward a future of intelligent, transparent, and efficient systems that will improve the world.

This is an exciting time, filled with both challenges and possibilities. With careful planning and a commitment to responsible innovation, non-transformer AI can illuminate the path toward a more adaptable and trustworthy digital era. Those who stake their claim now may find themselves leading a new wave of progress that transforms not just industries but entire communities and economies. By recognizing the value of transparency and the need for explainable decision-making, we can create an ecosystem where AI becomes a force for good—one that listens, learns, and grows alongside us.

For LPs: Investing in non-transformer AI models can be a strategic hedge against the risks of heavily betting on resource-intensive, mainstream AI approaches. First, these emerging technologies often come with smaller initial funding requirements, allowing LPs to diversify across multiple ventures rather than committing large sums to a single juggernaut. Second, non-transformer models tend to align with sustainability and regulatory trends, which can reduce reputational risks and attract environmentally conscious stakeholders. Finally, these investments may offer an earlier path to monetization since smaller, more adaptable AI models can move from proof-of-concept to market-ready solutions faster than massive transformer systems.

For VCs: Non-transformer models solve real-world pain points—lower energy usage, greater interpretability, and quicker deployment—that appeal to businesses facing cost or regulatory constraints. Second, backing startups in areas like neuromorphic hardware or liquid neural networks allows VCs to lead emerging markets where competition from tech giants is still manageable. Third, these models often excel in edge computing and time-sensitive applications, offering scalable revenue streams through licensing, subscription, or specialized hardware sales.

For Senior Execs: Non-transformer AI can provide a competitive edge without incurring large transformer models' extensive expenses or infrastructure overhauls. First, these technologies are better suited for real-time, adaptive scenarios such as robotics, personalized services, or supply chain optimization. Second, their focus on transparency and explainability can help meet mounting regulatory and consumer demands for trustworthy AI—vital for industries like finance and healthcare. Third, deploying more efficient architectures can lower operational costs and reduce carbon footprints, enhancing the bottom line and corporate social responsibility profiles.

For Founders: Pursuing non-transformer AI technologies can differentiate startups in a crowded field. First, demonstrating the ability to handle real-world challenges with minimal resources can attract venture interest even if you lack the capital to train massive transformer models. Second, emphasizing explainability and safety can boost customer adoption in critical sectors like legal tech, medicine, or autonomous systems, where trust is paramount. Finally, iterating rapidly on smaller, adaptive AI models allows quicker pivots and validation. This enables founders to refine their value proposition and stand out from competitors focused on large-scale, generalized AI.

Let’s Wrap This Up

Non-transformer AI is far more than a niche alternative to the dominant transformer-based approach. Its efficiency, adaptability, and focus on transparent decision-making present a compelling case for anyone looking to invest in or build the next generation of intelligent technology. Limited Partners find opportunities to diversify responsibly while focusing on long-term sustainability. Venture Capitalists see a chance to back startups whose unique solutions can carve out entirely new market segments. Senior Executives gain advanced capabilities without overhauling their infrastructure or violating newly emerging regulations. Founders stand out by delivering lean, trustworthy, and rapidly deployable products that resonate with clients and regulators alike.

As the AI landscape evolves, now is the time to pay attention to these non-transformer breakthroughs. They promise more efficient tools, novel applications, and a blueprint for responsible AI that benefits all stakeholders—from end users to corporate boards to the broader community. By embracing these next-generation models, the tech ecosystem can move closer to a future in which innovation, ethics, and profits go hand in hand.

Are You Ready To Implement AI?
Take The AI iQ Readiness Assessment
Get your results instantly

Qantm AI provides guidance on AI strategy and governance based on an approach developed over a decade of transforming Fortune 500 companies and creating tens of billions of dollars of value for them. We also provide education for senior leaders and various advisory services for private equity and venture capital firms.

RECENT PODCASTS:

🔊 BetterTech, hosted by Jocelyn Houle. LIVE ON December 4, 2024
🔊 AI and the Future of Work published November 4, 2024
🔊 Humain Podcast published September 19, 2024
🔊 Geeks Of The Valley. published September 15, 2024
🔊 HC Group published September 11, 2024
🔊 American Banker published September 10, 2024

UPCOMING EVENTS:

NASA Washington D.C. 25 Jan ‘25
Metro Connect USA 2025 Fort Lauderdale FL 24-26 Feb ‘25
2025: Paris, Milan, Hong Kong, New Delhi

Join 1000s of investors & startups at the Early-Stage Virtual Conf on Jan 30! 128 VCs, Family Offices, & angels committed to 1:1 Speed Pitch meetings with startups during the event alongside panels on VC trends 2025, how Family Offices invest in VC funds and startups, & best fundraising practices. Register for free

This conf helps investors: 1) become value-added investors by exploring those startups that need support beyond money, 2) add 1-3K startups in your funnel, & 3) find co-investors for your SPVs.

This conf helps startups to connect with active investors in your sector/vertical & finish your fundraising campaign.

More speakers/topics on the website. JAN 30, RSVP: