MeshAI
Benchmarks & Research

Evidence first. Results only when they exist.

An open benchmark programme for AI infrastructure researchers, hardware makers, model-serving teams, hyperscalers and universities.

SimulatedMeasured (lab)Independently reproduced
01Experiments

Planned experiments

No measured results have been published yet. Values appear here only once they exist, with provenance.

IDExperimentMetricHardwareRuntimeResultStatus
exp-001Quantisation vs quality for isiZulu & SwahiliModel quality preservationNVIDIA L40SvLLMNot yet measuredPlanned
exp-002Decode throughput across heterogeneous acceleratorsTokens per secondH100 / MI300X / M2 UltravLLM, llama.cppNot yet measuredPlanned
exp-003Power-aware scheduling under load-sheddingCost per successful taskNVIDIA L40SvLLMNot yet measuredPlanned
exp-004Connectivity-loss failover to local executionOutage recovery timeJetson AGX Orinllama.cppNot yet measuredPlanned
exp-005Energy efficiency by precisionTokens per jouleNVIDIA A100vLLMNot yet measuredPlanned
exp-006Tail latency under bursty demandP50 / P95 / P99 latencyNVIDIA H100vLLMNot yet measuredPlanned
exp-007Runtime compatibility matrixHardware compatibility8 profilesvLLM, Ollama, llama.cppNot yet measuredPlanned
02Benchmark categories

What we will measure

Tokens per secondtok/s
Time to first tokenms
P50 / P95 / P99 latencyms
Requests per secondreq/s
Cost per successful taskUSD
Tokens per jouletok/J
GPU utilisation%
Memory efficiency%
Outage recovery times
Network resilience% success
Model quality preservationΔ score
Hardware compatibilitypass/fail
03Methodology & reproducibility

How results will be produced

MeshAI Benchmark Methodology (draft v0.1)

1. Scope: inference workloads only; hardware, runtime, model, precision and driver versions recorded.
2. Warm-up: discard first 60 s of each run.
3. Load: fixed and bursty request profiles; input/output token distributions published.
4. Metrics: tok/s, TTFT, P50/P95/P99, req/s, cost per successful task, tok/J, utilisation,
   memory efficiency, outage recovery time, network resilience, quality preservation.
5. Energy: measured at the wall where possible; otherwise accelerator telemetry, labelled as such.
6. Quality: task-level evaluation, including African-language test sets, before and after optimisation.
7. Provenance: every result labelled Simulated, Measured (lab) or Independently reproduced.
8. Reproducibility: configs, seeds and scripts published with each result.

Note · Want to reproduce or contribute an experiment? Research partners can request access through the contact page.

More useful AI from every watt, GPU and dollar.

MeshAI is building the intelligence that makes AI infrastructure more efficient, resilient and economically accessible — starting where infrastructure constraints are most demanding.