- Home
- Products and Trials
- Product: HyperQ®
Hyper-state aware
reinforcement learning.
Traditional tabular RL needs you to enumerate the full state space up front. HyperQ doesn’t. It grows its state and action space as the model explores, learns across partial state transitions, and runs on CPU at scale — ready for production .NET workloads, with an open-source community edition you can clone today.
For engineering teams and applied ML practitioners building reinforcement-learning systems on .NET.
Beyond Ordinary HyperQ®
HyperQ® is our hyper-state aware tabular reinforcement learning framework. Written for .NET, it supports as much state as your memory will hold and can compute across multiple instances.
Traditional tabular reinforcement learning relies on a static enumeration of the state and action space — which limits Q-learning to environments small enough to fit fully in memory. We saw that as an opportunity. HyperQ is a first-of-its-kind hyper-state aware tabular learner: the framework expands its state as it explores the world, supports the addition of new actions, and uses a hyper-plane architecture for the Q matrix to enable transfer learning across partial state transitions in real time.
Your learning model can apply partial state-learning experiences to similar-but-new situations automatically — no extra work on your part, with high speed and accuracy.
Easy integration into your model
private void Step()
{
QState<decimal> state = _world.CurrentQState;
QAction a = _selector.SelectAction(_q, state, _hp.Epsilon);
decimal[] up = _world.Update(a);
_q.OffPolicyUpdate(state, a.Action, _world.CurrentQState, (double)up[1], _hp, EvalMethodType.Max);
_hp.Decay();
_rewardAverage.Add((double)up[1]);
_rewardHistory.Enqueue((double)_rewardAverage.Value);
if(_rewardHistory.Count > 50) _rewardHistory.Dequeue();
_epsilonHistory.Enqueue(_hp.Epsilon.Value);
if(_epsilonHistory.Count > 50) _epsilonHistory.Dequeue();
_alphaHistory.Enqueue(_hp.Alpha.Value);
if(_alphaHistory.Count > 50) _alphaHistory.Dequeue();
DrawGrid();
DrawActionLikelihoodGrid();
DrawLayeredQHeatMap();
DrawRewardGraph();
DrawParamGraph();
if(_world.Done)
{
_world.Reset();
ResetUI();
}
}You provide the model environment, we provide the learning. Browse the public sample repository: github.com/BeyondOrdinary/HyperQ-Samples.
Highly Capable and Extensible
- Hyper-plane state management and transfer learning across partial state transitions.
- Optimized for .NET.
- Start, pause, and resume training without losing progress.
- Real-time telemetry hooks for visualizing learning inside your model.
- Distributable across instances for high-memory-pressure models.
- No GPU required — pure CPU compute.
- Requires Microsoft's .NET Framework (not .NET Core).
Meet OpenHyperQ — HyperQ, freed.
OpenHyperQ is the community edition of our hyper-state aware reinforcement learning framework. We've taken the core learner libraries that power production HyperQ® deployments and released them under GPL‑3.0 so researchers, students, hobbyists, and curious engineers can build with the same ideas — no purchase order required.
Built fresh on .NET 10, OpenHyperQ ships source-first, sample-rich, and ready to run on any CPU. Clone it, train it, fork it, hack it — and when you're ready for enterprise-scale workloads, distributed compute, and direct access to our engineers, the commercial HyperQ® license below is waiting.
What's in the box
- Q learners:
ClassicQ,MappedQ<T>, plus double-Q variants for noisy reward landscapes. - HyperQ learners:
SingleHyperQ<T>,DoubleHyperQ<T>, and the experimentalLayeredHyperQ<T>for hierarchical state composition. - MultiHead learners: keep reward dimensions inspectable with pluggable scalarizers.
- MACE: multi-action collective trainer, evaluator, and memory.
- SARSA trainer with replay memory and Dyna support.
- Typed checkpoint API — no more
BinaryFormatter. - Runnable samples: GridWorld, LEM, HuntTheWumpus — with WinForms visualizers that let you watch the Q-table light up in real time.
Get started in two minutes
# Clone, build, learn
git clone https://github.com/BeyondOrdinary/OpenHyperQ.git
cd OpenHyperQ
dotnet build HyperQ.OSS.sln
# Watch a learner solve GridWorld
dotnet run --project samples/GridWorld.Viz/GridWorld.Viz.csproj
# Try the multi-feature Wumpus visualizer
dotnet run --project samples/Wumpus.Viz/Wumpus.Viz.csprojRequires the .NET 10 SDK. Windows is needed for the WinForms visualizers; the core libraries run anywhere .NET 10 runs.
OpenHyperQ vs. HyperQ® Commercial
| Capability | OpenHyperQ | HyperQ® Commercial |
|---|---|---|
| Core learners (Classic, Mapped, Hyper, MultiHead, MACE) | ||
| SARSA trainer, replay memory, Dyna support | ||
| CPU-only — no GPU required | ||
| .NET 10 source release | ||
| Distributed multi-instance training | — | |
| Production-hardened binaries & legacy project support | — | |
| Direct engineering support & model implementation help | Community | |
| Commercial-friendly licensing for proprietary products | GPL-3.0 only |
OpenHyperQ is distributed under the GNU General Public License v3.0. If you need to embed HyperQ in a closed-source product, ship distributed training, or get hands-on engineering support, see the commercial licensing details below.
HyperQ® commercial licensing
A subscription model that includes updates, technical support, documentation, and access to our engineers to help with your model implementation.
Licensing details
- .NET Framework
- Subscription license
- $18,999 per machine with unlimited CPUs, single legal entity.
- Premium support: 50% of license cost / year — direct engineer access.
- Standard tier: 25% of license cost / year — web docs and updates.
From GridWorld in 2 minutes to production RL.
OpenHyperQ runs on any .NET 10 machine in two commands. If you need distributed training, closed-source licensing, or direct engineering help on your model, 30 minutes is enough to walk through what the commercial edition looks like in your stack.