Apache ORC (Optimized Row Columnar)
Stripe-based columnar storage format originating in the Hive/Trino/Presto ecosystem with Protobuf-encoded Footer and a 1-byte PostScript length at the very last byte of the file.
Starts with 3-byte ASCII `ORC` (`4F 52 43`). The final byte (`EOF - 1`) gives the uncompressed PostScript length (always < 256 bytes).
Drop any .orc file (even 100 GB+) to inspect locally
Reads tiny File.slice() header/footer pages in a sandboxed Web Worker · 0 Bytes Uploaded
.orc Magic Bytes & Header Offset Map
3-byte ASCII 'ORC' file header
| Byte Offset | Length | Data Type | Field Name | Engineering Description |
|---|---|---|---|---|
| 0x00 - 0x02 | 3 B | char[3] | magic | ASCII 'ORC' header |
| 0x03 .. | var | stripes | stripe_data | Index, data, and stripe footer streams |
| EOF-1 | 1 B | uint8 | postscript_len | Byte length of uncompressed PostScript message |
.orc Security & Polyglot Risk Matrix
Check head[0:3] == 'ORC' and verify that the PostScript struct preceding EOF-1 ends with 'ORC'.
Read .orc Header & Metadata Without Heavy Dependencies
import os, sys
with open(sys.argv[1], "rb") as f:
assert f.read(3) == b"ORC"
f.seek(-1, os.SEEK_END)
ps_len = f.read(1)[0]
print(f"ORC valid: postscript_len={ps_len}B")Deterministic .orc Sample Files & Edge Cases
minimal-stripe.orc
Real ORC file written by pyarrow.orc: ORC magic, single stripe, Footer + PostScript trailer.
curl -fsSL https://fileformat.co/fixtures/v1/orc/minimal-stripe.orc -o minimal-stripe.orc \
&& echo "09713ef2158b8196359189b71dc332a3e57f8203547a2926a4bd3d0407f1ce0e minimal-stripe.orc" | shasum -a 256 -cwarehouse-stripe.orc
200-row ORC file from pyarrow.orc for Hive/Trino/Spark reader identification tests.
curl -fsSL https://fileformat.co/fixtures/v1/orc/warehouse-stripe.orc -o warehouse-stripe.orc \
&& echo "53afde4eadd2f23134d25a33b5b7418de973fc6996d3aeb39a20f2095b5e066c warehouse-stripe.orc" | shasum -a 256 -ctruncated-postscript.orc
ORC file missing the trailing PostScript tail — pyarrow ORCFile must raise.
curl -fsSL https://fileformat.co/fixtures/v1/orc/truncated-postscript.orc -o truncated-postscript.orc \
&& echo "8dd70b083ac96f1241bf50e890d1e5126bbd49910ad9cb927a00111ba693a2da truncated-postscript.orc" | shasum -a 256 -cRelated Specifications & Alternatives
De-facto standard hybrid columnar storage format for modern data lakes (Iceberg, Delta, DuckDB, Spark), placing Thrift Compact Protocol schema and statistics in the file trailer.
Row-oriented binary data serialization container widely used in Kafka/Confluent event streaming and Iceberg manifest lists, embedding full JSON writer schema in the file header.
Zero-copy in-memory columnar format serialized to disk with 8-byte aligned `ARROW1\x00\x00` framing and FlatBuffers schema/record-batch descriptors.