Beyond Ordinary HyperQ®

HyperQ® is our hyper-state aware tabular reinforcement learning framework. This framework is written for .NET applications, supports as much state as your memory can support, and is capable of computing across multiple instances.

Traditional tabular reinforcement learning uses a static enumeraiton of the state and action space. This limited the applicability of RL and Q learning to environments that could be enuemrated and fully realized in memory. Seeing this shortcoming as an opportunity, we created the first of its kind hyper-state aware tabular learner. Our RL framework will expand its state as it explores the world, supporting the addition of actions as well as state. Husing a hyper-plane architecture for the Q matrix, the framework is capable of transfer learning in real time across partial state transitions.

This means your learning model can utilize partial state learning experiences in similar but new experiences without extra work on your part. The framework will do this automatically and with incredible speed and accuracy.

Easy integration into your model

private void Step()
{
    QState<decimal> state = _world.CurrentQState;
    QAction a = _selector.SelectAction(_q, state, _hp.Epsilon);
    decimal[] up = _world.Update(a);
    _q.OffPolicyUpdate(state, a.Action, _world.CurrentQState, (double)up[1], _hp, EvalMethodType.Max);
    _hp.Decay();
    _rewardAverage.Add((double)up[1]);
    _rewardHistory.Enqueue((double)_rewardAverage.Value);
    if(_rewardHistory.Count>50) _rewardHistory.Dequeue();
    _epsilonHistory.Enqueue(_hp.Epsilon.Value);
    if(_epsilonHistory.Count>50) _epsilonHistory.Dequeue();
    _alphaHistory.Enqueue(_hp.Alpha.Value);
    if(_alphaHistory.Count>50) _alphaHistory.Dequeue();
    DrawGrid();
    DrawActionLikelihoodGrid();
    DrawLayeredQHeatMap();
    DrawRewardGraph();
    DrawParamGraph();
    if(_world.Done)
    {
        _world.Reset();
        ResetUI();
    }
}

You provide the model environment and we provide the learning. Check out the public repository of samples: https://github.com/BeyondOrdinary/HyperQ-Samples

Highly Capable and Extensible

Hyper-plane state management and transfer learning capabilities.
Optimized for .NET
Start, pause, and resume your learning experience.
Many real-time telemtry hooks to visualize learning in your model.
Distributed learning environment for high memory pressure models.
No GPU required. This is straight-up CPU compute.
Requires Microsoft's .NET Framework (Not core)

Beyond Ordinary HyperQ® Pricing

A subscription model where you get updates, support, how-to, and access to our staff to help with your model implementation.

Licensing Details

.NET Framework
Subscription License
$18,999 per machine with unlimited CPUs, single legal entity.
Wanna talk to a human? 50% of your licensed cost, per year, for that level of support.
OK with just web docs and getting updates? 25% of your licensed cost.

Beyond Ordinary HyperQ® Hyper-State Aware Reinforcement Learning