Reproducible Selectivity Study • OpenSearch 3.9.0 (Lucene 10.5.1 HNSW)

Filtered search selectivity: MassEngine vs. OpenSearch 3.9.

Apples-to-apples evaluation on identical 40-vCPU hosts (2 nodes × 20 cores, 8 primary shards, RF=2) across 2,000,000 and 20,000,000 1024-d English Wikipedia vectors (BAAI/bge-m3), sweeping deterministic filter selectivity from 100% down to 0.1%.

32.4× QPS

At 5% filter selectivity

6,993 QPS (1.15 ms) vs. 216 QPS (22.3 ms) on OpenSearch

7.0× less RAM

4.88 GiB vs. 34.1 GiB / node

Measured RSS on replicated 2M × 1024-d corpus

27.7 sec

Index build + replica sync

21× faster than OpenSearch bulk + HNSW force-merge

Want to reproduce this study on your own hosts? Redeem in the Console for a free 8-core, 21-day trial.

1. 2M-vector selectivity sweep & footprint (40 vCPUs, RF=2)

Each document is deterministically assigned grp = (id × 719) % 1000. Both engines ran with 8 primary shards, RF=2, 10 concurrent client threads, k=10, and exact FP32 brute-force ground truth (OpenSearch 3.9 force-merged to 1 segment/shard, m=16, ef_construction=100, 32 GiB JVM heap).

2,000,000 × 1024-d Wikipedia Selectivity Sweep (2 Nodes × 20 Cores, RF=2)
Filter Selectivity	MassEngine QPS	OpenSearch 3.9 QPS	MassEngine Latency	OpenSearch Latency	Recall@10 (ME / OS)	Speedup
Unfiltered (100%)	981.0 QPS	839.6 QPS	5.82 ms	13.56 ms	0.967 / 0.962	1.2× QPS • 2.3× faster
50% (grp ≤ 499)	1,938.5 QPS	761.6 QPS	3.41 ms	13.81 ms	0.965 / 0.982	2.5× QPS • 4.0× faster
10% (grp ≤ 99)	5,849.0 QPS	392.2 QPS	1.52 ms	18.51 ms	0.955 / 0.997	14.9× QPS • 12.2× faster
5% (grp ≤ 49)	6,992.5 QPS	216.0 QPS	1.15 ms	22.30 ms	0.952 / 1.000	32.4× QPS • 19.4× faster
1% (grp ≤ 9)	5,962.2 QPS	686.5 QPS	1.13 ms	13.57 ms	0.940 / 1.000	8.7× QPS • 12.0× faster
0.1% (grp ≤ 0)	5,387.3 QPS	935.2 QPS	0.91 ms	11.46 ms	0.930 / 1.000	5.8× QPS • 12.6× faster

Resource Footprint & Index Build Time (2M × 1024-d, 8 Shards, RF=2)
Resource Metric	MassEngine	OpenSearch 3.9.0 (Lucene 10.5.1)	Advantage
RAM per replicated node (RSS)	4.88 GiB / node	34.1 GiB / node	7.0× less RAM
Total index build + replica sync	27.7 sec (72,202 docs/s)	582.9 sec (93.2s bulk + 489.7s force-merge)	21.0× faster build
On-disk shard footprint per node	4.50 GiB / node	7.77 GiB / node	1.73× smaller disk

2. 20M-vector live cluster topologies

All three 20,000,000-vector Wikipedia clusters (83.46 GiB logical size, 8 shards) are running live and can be queried side-by-side at massengine.com/demo.

20,000,000 × 1024-d English Wikipedia Across 3 Live Topologies
Profile	Topology	Storage Mode	RAM / Node	Throughput & Latency	All-In Cost (VMs + License)
Setup 1: mem-2n	2 × 20 Cores (40 vCPUs, RF=2)	In-Memory	44.6 GiB	1,774 QPS (5.57 ms) • 14,580 QPS filtered	~$4,920 / mo
Setup 2: nvme-2n	2 × 20 Cores (40 vCPUs, RF=1)	Direct NVMe	2.52 GiB	316 QPS unfiltered • 2,115 QPS filtered (4.5 ms)	~$4,400 / mo
Setup 3: nvme-4n	4 × 1-vCPU Linux VMs (RF=1)	Direct NVMe	1.27 GiB	Sub-20ms interactive search • 100% Top-K match	$536 / mo (~$140 VMs + $396)

Inspect reproducible Python driver, OpenSearch mapping, CLI commands & raw JSON