feat(ast/estree): raw transfer (experimental) (#9516)

First version of raw transfer (#2409).

Provides a solid speed-up transferring data from Rust to JS. Further iterations will speed it up further.

Tests check that output via raw transfer matches output via JSON transfer exactly for all of Test262 which Acorn is able to parse. It should also match for TypeScript and JSX, but that's not covered by tests as yet.

However, I think we should consider this experimental for now because there are a few rough edges (discussed below).

Therefore I've put it behind an "experimental" flag:

```js
const ret = parseSync(filename, code, { experimentalRawTransfer: true });
console.log(ret.program);
```

### How it works

* JS creates an `ArrayBuffer` and passes it to Rust.
* Rust creates an `Allocator` using that buffer as its backing memory.
* Rust parses the AST into that allocator (including comments).
* Rust also converts module record and errors into arena types, and writes them into the allocator.
* Rust writes into the end of the buffer the offset at which data begins in the buffer.
* Control passes back to JS.
* JS code decodes data in the buffer, and creates JS objects fitting the ESTree shape.

There is *no* serialization step on Rust side. There is no JSON encoding or decoding involved. These are the main sources of the speed gain.

### Preconditions

The reason all this works is that all AST types (and other transferred types) are `#[repr(C)]`. So the memory layouts of those types are specified, and can be statically calculated.

`oxc_ast_tools` does those layout calculations, and generates the JS-side deserializer code based on its knowledge of what offsets struct fields are at, what the discriminants of enums are etc.

### Rough edges

There are a few rough edges.

#### `Allocator::from_raw_parts`

This PR adds a method `Allocator::from_raw_parts`. It creates a `bumpalo::Bump` that uses an existing block of memory as the allocator's backing memory. This ability is required, but it's not an API that `bumpalo` offers (and the maintainer was not willing to add it).

The implementation of `Allocator::from_raw_parts` is extremely hacky. It depends on internal implementation details of `bumpalo` which are not specified. And the method used to determine the memory layout of `Bump` depends on unspecified details of Rust's memory model. So, while it does seem to work in practice, it is, strictly speaking, UB.

It could break in a future Rust version, or with esoteric compiler flags e.g. [`-Zrandomize-layout`](https://doc.rust-lang.org/nightly/unstable-book/compiler-flags/randomize-layout.html). And it could break if we update `bumpalo` from the version we're currently using. For this reason, this PR pins the version of `bumpalo` in `Cargo.toml`.

I think this OK for now, but it's unpleasantly fragile.

We can resolve all these problems by replacing `bumpalo` with our own arena allocator (which I think we should do anyway, for other reasons).

In the meantime, `Allocator::from_raw_parts` is behind a cargo feature `from_raw_parts`, to avoid it being used anywhere else in our codebase.

#### Unspecified type layouts

As noted above, all AST types are `#[repr(C)]`, so their layout is specified and stable. There are a few types which are outside of our control, though:

1. `Vec<T>`. We use `allocator_api2::vec::Vec`, which is not `#[repr(C)]`.
2. `&str`. I don't believe the layout of this type is specified.
3. `Option<T>`.

For production-grade stability, we need to try to work around these.

`Vec` - we should replace `allocator_api2::vec::Vec` with our own `Vec` type. This will also allow us to reduce its size (https://github.com/oxc-project/backlog/issues/18).

`&str` - again, we need our own string slice type, to work around the problem of lone surrogates (#3526) and to make it more efficient (https://github.com/oxc-project/backlog/issues/46). We can make its memory layout stable at the same time.

`Option<T>` is tricky. We don't want to replace Rust's `Option` because of the niche optimization benefits it gives. I'm not sure this one is 100% soluable, but Rust gives at least *some* guarantees about the layout of `Option`. Maybe we can avoid using `Option` in the AST in ways which go outside that specification.

#### Large buffers

For speed, raw transfer requires the entire AST to be in a single contiguous memory region, and for the start of that region to be aligned on 4 GiB.

JS does not support 64-bit integers, so offset calculations are much cheaper when the buffer is aligned on 4 GiB and no larger than 4 GiB - because then all pointers have the same value in their top 32 bits. So the pointer can be treated as a 32-bit value (bottom 32 bits only). JS can handle 32 bit integers no problem.

When creating a large buffer on JS side, it *mostly* ends up aligned on a 4 GiB boundary anyway, but occasionally it doesn't. So in order to ensure the buffer has at least 1 region within it which is aligned on 4 GiB, and 2 GiB in size, we have to create a 6 GiB buffer.

I *think* this is OK. On systems with virtual memory, allocating 6 GiB only reserves 6 GiB of *virtual* memory. Physical memory is only consumed when the pages of that allocation are actually written to.

But I may be missing something here, and memory exhaustion might be a danger. I think we need some real-world usage to find out.

*Possibly* we could reduce the need for so much memory if JS deserializer called into a small WASM module to do offset calculations. WASM can work with `i64` values. Or there may be other solutions.

#### Endianness

Currently only little-endian systems are supported. Probably in practice this doesn't matter much, but it'd be ideal to cover big-endian too.

.github/.generated_ast_watch_list.yml

@@ -39,10 +39,16 @@ src:
   - 'crates/oxc_syntax/src/generated/derive_content_eq.rs'
   - 'crates/oxc_syntax/src/generated/derive_estree.rs'
   - 'crates/oxc_syntax/src/lib.rs'
+  - 'crates/oxc_syntax/src/module_record.rs'
   - 'crates/oxc_syntax/src/number.rs'
   - 'crates/oxc_syntax/src/operator.rs'
   - 'crates/oxc_syntax/src/reference.rs'
   - 'crates/oxc_syntax/src/scope.rs'
+  - 'crates/oxc_syntax/src/serialize.rs'
   - 'crates/oxc_syntax/src/symbol.rs'
+  - 'napi/parser/deserialize.js'
+  - 'napi/parser/src/generated/assert_layouts.rs'
+  - 'napi/parser/src/generated/derive_estree.rs'
+  - 'napi/parser/src/raw_transfer_types.rs'
   - 'npm/oxc-types/types.d.ts'
   - 'tasks/ast_tools/src/**'

.github/workflows/ci.yml

@@ -303,6 +303,8 @@ jobs:
           filters: |
             src:
               - '!crates/oxc_linter/**'
+      - uses: ./.github/actions/clone-submodules
+        if: steps.filter.outputs.src == 'true'
       - uses: oxc-project/setup-rust@cd82e1efec7fef815e2c23d296756f31c7cdc03d # v1.0.0
         if: steps.filter.outputs.src == 'true'
         with:
@@ -311,7 +313,7 @@ jobs:
         if: steps.filter.outputs.src == 'true'
       - if: steps.filter.outputs.src == 'true'
         run: |
-          pnpm run build-dev
+          pnpm run build
           pnpm run test
           pnpm --filter e2e run test
           git diff --exit-code # Must commit everything

Cargo.lock

@@ -1994,6 +1994,8 @@ dependencies = [
  "napi-build",
  "napi-derive",
  "oxc",
+ "oxc_ast_macros",
+ "oxc_estree",
  "oxc_napi",
  "rustc-hash",
 ]

Cargo.toml

@@ -160,7 +160,11 @@ assert-unchecked = "0.1.2"
 base64 = "0.22.1"
 bitflags = "2.8.0"
 bpaf = "0.9.16"
-bumpalo = "3.17.0"
+# `bumpalo` must be pinned to exactly this version.
+# `Allocator::from_raw_parts` (used in raw transfer) depends on internal implementation details
+# of `bumpalo` which may change in a future version.
+# This is a temporary situation - we'll replace `bumpalo` with our own allocator.
+bumpalo = "=3.17.0"
 compact_str = "0.8.1"
 console = "0.15.10"
 console_error_panic_hook = "0.1.7"

crates/oxc/Cargo.toml

@@ -65,6 +65,7 @@ isolated_declarations = ["oxc_isolated_declarations"]
 ast_visit = ["oxc_ast_visit"]
 
 serialize = [
+  "oxc_allocator/from_raw_parts",
   "oxc_ast/serialize",
   "oxc_ast_visit/serialize",
   "oxc_semantic?/serialize",

crates/oxc_allocator/Cargo.toml

@@ -34,4 +34,5 @@ serde = { workspace = true }
 serde_json = { workspace = true }
 
 [features]
+from_raw_parts = []
 serialize = ["dep:serde", "oxc_estree/serialize"]

crates/oxc_allocator/src/allocator.rs

@@ -435,6 +435,18 @@ impl Allocator {
     pub(crate) fn bump(&self) -> &Bump {
         &self.bump
     }
+
+    /// Create [`Allocator`] from a [`bumpalo::Bump`].
+    ///
+    /// This method is not public. Only used by [`Allocator::from_raw_parts`].
+    //
+    // `#[inline(always)]` because it's a no-op
+    #[cfg(feature = "from_raw_parts")]
+    #[expect(clippy::inline_always)]
+    #[inline(always)]
+    pub(crate) fn from_bump(bump: Bump) -> Self {
+        Self { bump }
+    }
 }
 
 /// SAFETY: Not actually safe, but for enabling `Send` for downstream crates.

crates/oxc_allocator/src/from_raw_parts.rs

@@ -0,0 +1,206 @@
+//! Define additional [`Allocator::from_raw_parts`] method, used only by raw transfer.
+
+use std::{
+    alloc::Layout,
+    cell::Cell,
+    ptr::{self, NonNull},
+};
+
+use bumpalo::Bump;
+
+use crate::Allocator;
+
+/// Minimum alignment for allocator chunks. This is hard-coded on `bumpalo`.
+const MIN_ALIGN: usize = 16;
+
+const CHUNK_FOOTER_SIZE: usize = size_of::<ChunkFooter>();
+const _: () = {
+    assert!(CHUNK_FOOTER_SIZE >= MIN_ALIGN);
+    assert!(align_of::<ChunkFooter>() <= MIN_ALIGN);
+};
+
+impl Allocator {
+    /// Minimum size for memory chunk passed to [`Allocator::from_raw_parts`].
+    pub const RAW_MIN_SIZE: usize = CHUNK_FOOTER_SIZE;
+
+    /// Minimum alignment for memory chunk passed to [`Allocator::from_raw_parts`].
+    pub const RAW_MIN_ALIGN: usize = MIN_ALIGN;
+
+    /// Construct a static-sized [`Allocator`] from an existing memory allocation.
+    ///
+    /// **IMPORTANT: WE MUST NOT CHANGE THE VERSION OF BUMPALO DEPENDENCY**.
+    ///
+    /// This code only remains sound as long as the code in version of `bumpalo` we're using matches
+    /// the duplicate of `bumpalo`'s internals contained in this file.
+    ///
+    /// `bumpalo` is pinned to version `=3.17.0` in `Cargo.toml`.
+    ///
+    /// The [`Allocator`] which is returned takes ownership of the memory allocation,
+    /// and the allocation will be freed if the `Allocator` is dropped.
+    /// If caller wishes to prevent that happening, they must wrap the `Allocator` in `ManuallyDrop`.
+    ///
+    /// The [`Allocator`] returned by this function cannot grow.
+    ///
+    /// This hack is all very inadvisable!
+    /// Only implemented as a temporary stopgap until we replace `bumpalo` with our own allocator.
+    ///
+    /// # SAFETY
+    ///
+    /// * `ptr` must be aligned on [`RAW_MIN_ALIGN`].
+    /// * `size` must be a multiple of [`RAW_MIN_ALIGN`].
+    /// * `size` must be at least [`RAW_MIN_SIZE`].
+    /// * The memory region starting at `ptr` and encompassing `size` bytes must be within
+    ///   a single allocation.
+    ///
+    /// # Panics
+    ///
+    /// Panics if cannot determine layout of Bumpalo's `Bump` type.
+    ///
+    /// [`RAW_MIN_ALIGN`]: Self::RAW_MIN_ALIGN
+    /// [`RAW_MIN_SIZE`]: Self::RAW_MIN_SIZE
+    pub unsafe fn from_raw_parts(ptr: NonNull<u8>, size: usize) -> Self {
+        // Only support little-endian systems.
+        // Calculating offset of `current_chunk_footer` on big-endian systems would be difficult.
+        #[cfg(target_endian = "big")]
+        const {
+            panic!("`Allocator::from_raw_parts` is not supported on big-endian systems.");
+        }
+
+        // Debug assert that `ptr` and `size` fulfill size and alignment requirements
+        debug_assert!(is_multiple_of(ptr.as_ptr() as usize, MIN_ALIGN));
+        debug_assert!(is_multiple_of(size, MIN_ALIGN));
+        debug_assert!(size >= CHUNK_FOOTER_SIZE);
+
+        // `Bump` is defined as:
+        //
+        // ```
+        // pub struct Bump {
+        //     current_chunk_footer: Cell<NonNull<ChunkFooter>>,
+        //     allocation_limit: Cell<Option<usize>>,
+        // }
+        // ```
+        //
+        // `Bump` is not `#[repr(C)]`, so which order the fields are in is unpredictable.
+        // Deduce the offset of `current_chunk_footer` field by creating a dummy `Bump` where the value
+        // of the `allocation_limit` field is known.
+        //
+        // This should all be const-folded down by compiler.
+        let current_chunk_footer_field_offset: usize = {
+            const {
+                assert!(size_of::<Bump>() == size_of::<[usize; 3]>());
+                assert!(align_of::<Bump>() == align_of::<[usize; 3]>());
+                assert!(size_of::<Cell<NonNull<ChunkFooter>>>() == size_of::<usize>());
+                assert!(align_of::<Cell<NonNull<ChunkFooter>>>() == align_of::<usize>());
+                assert!(size_of::<Cell<Option<usize>>>() == size_of::<[usize; 2]>());
+                assert!(align_of::<Cell<Option<usize>>>() == align_of::<usize>());
+            }
+
+            let bump = Bump::new();
+            bump.set_allocation_limit(Some(123));
+
+            // SAFETY:
+            // `Bump` has same layout as `[usize; 3]` (checked by const assertions above).
+            // Strictly speaking, reading the fields as `usize`s is UB, as the layout of `Option`
+            // is not specified. But in practice, `Option` stores its discriminant before its payload,
+            // so either field order means 3rd `usize` is fully initialized
+            // (it's either `NonNull<ChunkFooter>>` or the `usize` in `Option<usize>`).
+            // Once we've figured out the field order, should be safe to check the `Option`
+            // discriminant as a `u8`.
+            // Const assertion at top of this function ensures this is a little-endian system,
+            // so first byte of the 8 bytes containing the discriminant will be initialized, regardless
+            // of whether compiler chooses to represent the discriminant as `u8`, `u16`, `u32` or `u64`.
+            unsafe {
+                let ptr = ptr::from_ref(&bump).cast::<usize>();
+                if *ptr.add(2) == 123 {
+                    // `allocation_limit` is 2nd field. So `current_chunk_footer` is 1st.
+                    assert_eq!(*ptr.add(1).cast::<u8>(), 1);
+                    0
+                } else {
+                    // `allocation_limit` is 1st field. So `current_chunk_footer` is 2nd.
+                    assert_eq!(*ptr.add(1), 123);
+                    assert_eq!(*ptr.cast::<u8>(), 1);
+                    2
+                }
+            }
+        };
+
+        // Create empty bump with allocation limit of 0 - i.e. it cannot grow.
+        // This means that the memory chunk we're about to add to the `Bump` will remain its only chunk.
+        // Therefore it can never be deallocated, until the `Allocator` is dropped.
+        // `Allocator::reset` would only reset the "cursor" pointer, not deallocate the memory.
+        let bump = Bump::new();
+        bump.set_allocation_limit(Some(0));
+
+        // Get pointer to `EmptyChunkFooter`.
+        // SAFETY: We've established the offset of the `current_chunk_footer` field above.
+        let current_chunk_footer_field = unsafe {
+            let field_ptr = ptr::addr_of!(bump)
+                .cast::<Cell<NonNull<ChunkFooter>>>()
+                .add(current_chunk_footer_field_offset);
+            &*field_ptr
+        };
+        let empty_chunk_footer_ptr = current_chunk_footer_field.get();
+
+        // Construct `ChunkFooter` and write into end of allocation.
+        // SAFETY: Caller guarantees:
+        // 1. `ptr` is the start of an allocation of `size` bytes.
+        // 2. `size` is `>= CHUNK_FOOTER_SIZE` - so `size - CHUNK_FOOTER_SIZE` cannot wrap around.
+        let chunk_footer_ptr = unsafe { ptr.add(size - CHUNK_FOOTER_SIZE) };
+        // SAFETY: Caller guarantees `size` is a multiple of 16
+        let layout = unsafe { Layout::from_size_align_unchecked(size, 16) };
+        let chunk_footer = ChunkFooter {
+            data: ptr,
+            layout,
+            prev: Cell::new(empty_chunk_footer_ptr),
+            ptr: Cell::new(chunk_footer_ptr),
+            allocated_bytes: 0,
+        };
+        let chunk_footer_ptr = chunk_footer_ptr.cast::<ChunkFooter>();
+        // SAFETY: If caller has upheld safety requirements, `chunk_footer_ptr` is `CHUNK_FOOTER_SIZE`
+        // bytes from the end of the allocation, and aligned on 16.
+        // Const assertions at top of this file ensure that is sufficient alignment for `ChunkFooter`.
+        // Therefore `chunk_footer_ptr` is valid for writing a `ChunkFooter`.
+        unsafe { chunk_footer_ptr.write(chunk_footer) };
+
+        // Write chunk header into bump's `chunk_header` field
+        current_chunk_footer_field.set(chunk_footer_ptr);
+
+        Self::from_bump(bump)
+    }
+}
+
+/// Allocator chunk footer.
+///
+/// Copied exactly from `bumpalo` v3.17.0.
+///
+/// This type is not exposed by `bumpalo` crate, but the type is `#[repr(C)]`, so we can rely on our
+/// duplicate here having the same layout, as long as we don't change the version of `bumpalo` we use.
+#[repr(C)]
+#[derive(Debug)]
+struct ChunkFooter {
+    /// Pointer to the start of this chunk allocation.
+    /// This footer is always at the end of the chunk.
+    data: NonNull<u8>,
+
+    /// The layout of this chunk's allocation.
+    layout: Layout,
+
+    /// Link to the previous chunk.
+    ///
+    /// Note that the last node in the `prev` linked list is the canonical empty
+    /// chunk, whose `prev` link points to itself.
+    prev: Cell<NonNull<ChunkFooter>>,
+
+    /// Bump allocation finger that is always in the range `self.data..=self`.
+    ptr: Cell<NonNull<u8>>,
+
+    /// The bytes allocated in all chunks so far.
+    /// The canonical empty chunk has a size of 0 and for all other chunks, `allocated_bytes` will be
+    /// the allocated_bytes of the current chunk plus the allocated bytes of the `prev` chunk.
+    allocated_bytes: usize,
+}
+
+/// Returns `true` if `n` is a multiple of `divisor`.
+const fn is_multiple_of(n: usize, divisor: usize) -> bool {
+    n % divisor == 0
+}

crates/oxc_allocator/src/lib.rs

@@ -20,6 +20,8 @@ mod allocator_api2;
 mod boxed;
 mod clone_in;
 mod convert;
+#[cfg(feature = "from_raw_parts")]
+mod from_raw_parts;
 pub mod hash_map;
 pub mod string;
 mod vec;

crates/oxc_ast/src/ast/comment.rs

@@ -1,12 +1,14 @@
 #![warn(missing_docs)]
 use oxc_allocator::CloneIn;
 use oxc_ast_macros::ast;
+use oxc_estree::ESTree;
 use oxc_span::{ContentEq, Span};
 
 /// Indicates a line or block comment.
 #[ast]
-#[generate_derive(CloneIn, ContentEq)]
+#[generate_derive(CloneIn, ContentEq, ESTree)]
 #[derive(Debug, Default, Clone, Copy, Eq, PartialEq)]
+#[estree(no_rename_variants, no_ts_def)]
 pub enum CommentKind {
     /// Line comment
     #[default]
@@ -77,8 +79,9 @@ pub enum CommentAnnotation {
 
 /// A comment in source code.
 #[ast]
-#[generate_derive(CloneIn, ContentEq)]
+#[generate_derive(CloneIn, ContentEq, ESTree)]
 #[derive(Debug, Default, Clone, Copy, Eq, PartialEq)]
+#[estree(add_fields(value = CommentValue), field_order(kind, value, span), no_ts_def)]
 pub struct Comment {
     /// The span of the comment text, with leading and trailing delimiters.
     pub span: Span,
@@ -87,22 +90,28 @@ pub struct Comment {
     /// `/* Leading */ token`
     ///                ^ This start
     /// NOTE: Trailing comment attachment is not computed yet.
+    #[estree(skip)]
     pub attached_to: u32,
 
     /// Line or block comment
+    #[estree(rename = "type")]
     pub kind: CommentKind,
 
     /// Leading or trailing comment
+    #[estree(skip)]
     pub position: CommentPosition,
 
     /// Whether this comment has a preceding newline.
     /// Used to avoid becoming a trailing comment in codegen.
+    #[estree(skip)]
     pub preceded_by_newline: bool,
 
     /// Whether this comment has a tailing newline.
+    #[estree(skip)]
     pub followed_by_newline: bool,
 
     /// Comment Annotation
+    #[estree(skip)]
     pub annotation: CommentAnnotation,
 }
 

crates/oxc_ast/src/generated/derive_estree.rs

@@ -8,6 +8,7 @@ use oxc_estree::{
     ser::{AppendTo, AppendToConcat},
 };
 
+use crate::ast::comment::*;
 use crate::ast::js::*;
 use crate::ast::jsx::*;
 use crate::ast::literal::*;
@@ -3338,3 +3339,23 @@ impl ESTree for JSDocUnknownType {
         state.end();
     }
 }
+
+impl ESTree for CommentKind {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        match self {
+            Self::Line => JsonSafeString("Line").serialize(serializer),
+            Self::Block => JsonSafeString("Block").serialize(serializer),
+        }
+    }
+}
+
+impl ESTree for Comment {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        let mut state = serializer.serialize_struct();
+        state.serialize_field("type", &self.kind);
+        state.serialize_field("value", &crate::serialize::CommentValue(self));
+        state.serialize_field("start", &self.span.start);
+        state.serialize_field("end", &self.span.end);
+        state.end();
+    }
+}

crates/oxc_ast/src/serialize.rs

@@ -75,7 +75,7 @@ impl Program<'_> {
 
 /// Serialized as `null`.
 #[ast_meta]
-#[estree(ts_type = "null")]
+#[estree(ts_type = "null", raw_deser = "null")]
 pub struct Null<'b, T>(#[expect(dead_code)] pub &'b T);
 
 impl<T> ESTree for Null<'_, T> {
@@ -86,7 +86,7 @@ impl<T> ESTree for Null<'_, T> {
 
 /// Serialized as `true`.
 #[ast_meta]
-#[estree(ts_type = "true")]
+#[estree(ts_type = "true", raw_deser = "true")]
 pub struct True<'b, T>(#[expect(dead_code)] pub &'b T);
 
 impl<T> ESTree for True<'_, T> {
@@ -97,7 +97,7 @@ impl<T> ESTree for True<'_, T> {
 
 /// Serialized as `false`.
 #[ast_meta]
-#[estree(ts_type = "false")]
+#[estree(ts_type = "false", raw_deser = "false")]
 pub struct False<'b, T>(#[expect(dead_code)] pub &'b T);
 
 impl<T> ESTree for False<'_, T> {
@@ -108,7 +108,7 @@ impl<T> ESTree for False<'_, T> {
 
 /// Serialized as `"in"`.
 #[ast_meta]
-#[estree(ts_type = "'in'")]
+#[estree(ts_type = "'in'", raw_deser = "'in'")]
 pub struct In<'b, T>(#[expect(dead_code)] pub &'b T);
 
 impl<T> ESTree for In<'_, T> {
@@ -119,7 +119,7 @@ impl<T> ESTree for In<'_, T> {
 
 /// Serialized as `"init"`.
 #[ast_meta]
-#[estree(ts_type = "'init'")]
+#[estree(ts_type = "'init'", raw_deser = "'init'")]
 pub struct Init<'b, T>(#[expect(dead_code)] pub &'b T);
 
 impl<T> ESTree for Init<'_, T> {
@@ -134,7 +134,10 @@ impl<T> ESTree for Init<'_, T> {
 
 /// Serializer for `raw` field of `BooleanLiteral`.
 #[ast_meta]
-#[estree(ts_type = "string | null")]
+#[estree(
+    ts_type = "string | null",
+    raw_deser = "(THIS.start === 0 && THIS.end === 0) ? null : THIS.value + ''"
+)]
 pub struct BooleanLiteralRaw<'b>(pub &'b BooleanLiteral);
 
 impl ESTree for BooleanLiteralRaw<'_> {
@@ -153,7 +156,10 @@ impl ESTree for BooleanLiteralRaw<'_> {
 
 /// Serializer for `raw` field of `NullLiteral`.
 #[ast_meta]
-#[estree(ts_type = "'null' | null")]
+#[estree(
+    ts_type = "'null' | null",
+    raw_deser = "(THIS.start === 0 && THIS.end === 0) ? null : 'null'"
+)]
 pub struct NullLiteralRaw<'b>(pub &'b NullLiteral);
 
 impl ESTree for NullLiteralRaw<'_> {
@@ -166,7 +172,7 @@ impl ESTree for NullLiteralRaw<'_> {
 
 /// Serializer for `bigint` field of `BigIntLiteral`.
 #[ast_meta]
-#[estree(ts_type = "string")]
+#[estree(ts_type = "string", raw_deser = "THIS.raw.slice(0, -1).replace(/_/g, '')")]
 pub struct BigIntLiteralBigint<'a, 'b>(pub &'b BigIntLiteral<'a>);
 
 impl ESTree for BigIntLiteralBigint<'_, '_> {
@@ -180,7 +186,7 @@ impl ESTree for BigIntLiteralBigint<'_, '_> {
 ///
 /// Serialized as `null` in JSON, but updated on JS side to contain a `BigInt`.
 #[ast_meta]
-#[estree(ts_type = "BigInt")]
+#[estree(ts_type = "BigInt", raw_deser = "BigInt(THIS.bigint)")]
 pub struct BigIntLiteralValue<'a, 'b>(#[expect(dead_code)] pub &'b BigIntLiteral<'a>);
 
 impl ESTree for BigIntLiteralValue<'_, '_> {
@@ -191,7 +197,33 @@ impl ESTree for BigIntLiteralValue<'_, '_> {
 
 /// Serializer for `regex` field of `RegExpLiteral`.
 #[ast_meta]
-#[estree(ts_type = "RegExp")]
+#[estree(
+    ts_type = "RegExp",
+    raw_deser = r#"
+        let pattern, flags, value = null;
+        if (THIS.raw === null) {
+            pattern = DESER[RegExpPattern](POS_OFFSET.regex.pattern);
+            const flagBits = DESER[u8](POS_OFFSET.regex.flags);
+            flags = '';
+            if (flagBits & 1) flags += 'g';
+            if (flagBits & 2) flags += 'i';
+            if (flagBits & 4) flags += 'm';
+            if (flagBits & 8) flags += 's';
+            if (flagBits & 16) flags += 'u';
+            if (flagBits & 32) flags += 'y';
+            if (flagBits & 64) flags += 'd';
+            if (flagBits & 128) flags += 'v';
+        } else {
+            [, pattern, flags] = THIS.raw.match(/^\/(.*)\/([a-z]*)$/);
+        }
+
+        try {
+            value = new RegExp(pattern, flags);
+        } catch (e) {}
+
+        { pattern, flags }
+    "#
+)]
 pub struct RegExpLiteralRegex<'a, 'b>(pub &'b RegExpLiteral<'a>);
 
 impl ESTree for RegExpLiteralRegex<'_, '_> {
@@ -218,7 +250,11 @@ impl ESTree for RegExpLiteralRegex<'_, '_> {
 ///
 /// Serialized as `null` in JSON, but updated on JS side to contain a `RegExp` if the regexp is valid.
 #[ast_meta]
-#[estree(ts_type = "RegExp | null")]
+#[estree(
+    ts_type = "RegExp | null",
+    // `value` is defined by `RegExpLiteralRegex` converter
+    raw_deser = "value",
+)]
 pub struct RegExpLiteralValue<'a, 'b>(#[expect(dead_code)] pub &'b RegExpLiteral<'a>);
 
 impl ESTree for RegExpLiteralValue<'_, '_> {
@@ -253,7 +289,7 @@ impl ESTree for RegExpFlagsConverter<'_> {
 
 /// Serialize `ArrayExpressionElement::Elision` variant as `null`.
 #[ast_meta]
-#[estree(ts_type = "null")]
+#[estree(ts_type = "null", raw_deser = "null")]
 pub struct ElisionConverter<'b>(#[expect(dead_code)] pub &'b Elision);
 
 impl ESTree for ElisionConverter<'_> {
@@ -265,7 +301,26 @@ impl ESTree for ElisionConverter<'_> {
 /// Serialize `FormalParameters`, to be estree compatible, with `items` and `rest` fields combined
 /// and `argument` field flattened.
 #[ast_meta]
-#[estree(ts_type = "ParamPattern[]")]
+#[estree(
+    ts_type = "ParamPattern[]",
+    raw_deser = "
+        const params = DESER[Vec<FormalParameter>](POS_OFFSET.items);
+        if (uint32[(POS_OFFSET.rest) >> 2] !== 0 && uint32[(POS_OFFSET.rest + 4) >> 2] !== 0) {
+            pos = uint32[(POS_OFFSET.rest) >> 2];
+            params.push({
+                type: 'RestElement',
+                start: DESER[u32]( POS_OFFSET<BindingRestElement>.span.start ),
+                end: DESER[u32]( POS_OFFSET<BindingRestElement>.span.end ),
+                argument: DESER[BindingPatternKind]( POS_OFFSET<BindingRestElement>.argument.kind ),
+                typeAnnotation: DESER[Option<Box<TSTypeAnnotation>>](
+                    POS_OFFSET<BindingRestElement>.argument.type_annotation
+                ),
+                optional: DESER[bool]( POS_OFFSET<BindingRestElement>.argument.optional ),
+            });
+        }
+        params
+    "
+)]
 pub struct FormalParametersConverter<'a, 'b>(pub &'b FormalParameters<'a>);
 
 impl ESTree for FormalParametersConverter<'_, '_> {
@@ -303,7 +358,14 @@ impl ESTree for FormalParametersRest<'_, '_> {
 ///
 /// Serialize `specifiers` as an empty array if it's `None`.
 #[ast_meta]
-#[estree(ts_type = "Array<ImportDeclarationSpecifier>")]
+#[estree(
+    ts_type = "Array<ImportDeclarationSpecifier>",
+    raw_deser = "
+        let specifiers = DESER[Option<Vec<ImportDeclarationSpecifier>>](POS_OFFSET.specifiers);
+        if (specifiers === null) specifiers = [];
+        specifiers
+    "
+)]
 pub struct ImportDeclarationSpecifiers<'a, 'b>(pub &'b ImportDeclaration<'a>);
 
 impl ESTree for ImportDeclarationSpecifiers<'_, '_> {
@@ -318,6 +380,20 @@ impl ESTree for ImportDeclarationSpecifiers<'_, '_> {
 
 /// Serialize `ObjectProperty` with fields in same order as Acorn.
 #[ast_meta]
+#[estree(raw_deser = "
+    const start = DESER[u32](POS_OFFSET.span.start),
+        end = DESER[u32](POS_OFFSET.span.end),
+        method = DESER[bool](POS_OFFSET.method),
+        shorthand = DESER[bool](POS_OFFSET.shorthand),
+        computed = DESER[bool](POS_OFFSET.computed),
+        key = DESER[PropertyKey](POS_OFFSET.key),
+        kind = DESER[PropertyKind](POS_OFFSET.kind),
+        value = DESER[Expression](POS_OFFSET.value),
+        obj = method || shorthand || kind !== 'init'
+            ? {type: 'Property', start, end, method, shorthand, computed, key, kind, value}
+            : {type: 'Property', start, end, method, shorthand, computed, key, value, kind};
+    obj
+")]
 pub struct ObjectPropertyConverter<'a, 'b>(pub &'b ObjectProperty<'a>);
 
 impl ESTree for ObjectPropertyConverter<'_, '_> {
@@ -345,6 +421,18 @@ impl ESTree for ObjectPropertyConverter<'_, '_> {
 
 /// Serialize `BindingProperty` with fields in same order as Acorn.
 #[ast_meta]
+#[estree(raw_deser = "
+    const start = DESER[u32](POS_OFFSET.span.start),
+        end = DESER[u32](POS_OFFSET.span.end),
+        shorthand = DESER[bool](POS_OFFSET.shorthand),
+        computed = DESER[bool](POS_OFFSET.computed),
+        key = DESER[PropertyKey](POS_OFFSET.key),
+        value = DESER[BindingPattern](POS_OFFSET.value),
+        obj = shorthand
+            ? {type: 'Property', start, end, method: false, shorthand, computed, key, kind: 'init', value}
+            : {type: 'Property', start, end, method: false, shorthand, computed, key, value, kind: 'init'};
+    obj
+")]
 pub struct BindingPropertyConverter<'a, 'b>(pub &'b BindingProperty<'a>);
 
 impl ESTree for BindingPropertyConverter<'_, '_> {
@@ -375,7 +463,13 @@ impl ESTree for BindingPropertyConverter<'_, '_> {
 /// Serializes as either an expression (if `expression` property is set),
 /// or a `BlockStatement` (if it's not).
 #[ast_meta]
-#[estree(ts_type = "FunctionBody | Expression")]
+#[estree(
+    ts_type = "FunctionBody | Expression",
+    raw_deser = "
+        let body = DESER[Box<FunctionBody>](POS_OFFSET.body);
+        DESER[bool](POS_OFFSET.expression) ? body.body[0].expression : body
+    "
+)]
 pub struct ArrowFunctionExpressionBody<'a>(pub &'a ArrowFunctionExpression<'a>);
 
 impl ESTree for ArrowFunctionExpressionBody<'_> {
@@ -391,7 +485,23 @@ impl ESTree for ArrowFunctionExpressionBody<'_> {
 /// Serializer for `AssignmentTargetPropertyIdentifier`'s `init` field
 /// (which is renamed to `value` in ESTree AST).
 #[ast_meta]
-#[estree(ts_type = "IdentifierReference | AssignmentTargetWithDefault")]
+#[estree(
+    ts_type = "IdentifierReference | AssignmentTargetWithDefault",
+    raw_deser = "
+        const init = DESER[Option<Expression>](POS_OFFSET.init),
+            binding = DESER[IdentifierReference](POS_OFFSET.binding),
+            value = init === null
+                ? binding
+                : {
+                    type: 'AssignmentPattern',
+                    start: DESER[u32](POS_OFFSET.span.start),
+                    end: DESER[u32](POS_OFFSET.span.end),
+                    left: binding,
+                    right: init,
+                };
+        value
+    "
+)]
 pub struct AssignmentTargetPropertyIdentifierValue<'a>(
     pub &'a AssignmentTargetPropertyIdentifier<'a>,
 );
@@ -417,7 +527,13 @@ impl ESTree for AssignmentTargetPropertyIdentifierValue<'_> {
 ///
 /// Serialize only the first expression in `arguments`, or `null` if `arguments` is empty.
 #[ast_meta]
-#[estree(ts_type = "Expression | null")]
+#[estree(
+    ts_type = "Expression | null",
+    raw_deser = "
+        const args = DESER[Vec<Expression>](POS_OFFSET.arguments);
+        args.length === 0 ? null : args[0]
+    "
+)]
 pub struct ImportExpressionArguments<'a>(pub &'a ImportExpression<'a>);
 
 impl ESTree for ImportExpressionArguments<'_> {
@@ -435,6 +551,25 @@ impl ESTree for ImportExpressionArguments<'_> {
 /// Omit `with_clause` field (which is renamed to `attributes` in ESTree)
 /// unless `source` field is `Some`.
 #[ast_meta]
+#[estree(raw_deser = "
+    const start = DESER[u32](POS_OFFSET.span.start),
+        end = DESER[u32](POS_OFFSET.span.end),
+        declaration = DESER[Option<Declaration>](POS_OFFSET.declaration),
+        specifiers = DESER[Vec<ExportSpecifier>](POS_OFFSET.specifiers),
+        source = DESER[Option<StringLiteral>](POS_OFFSET.source),
+        exportKind = DESER[ImportOrExportKind](POS_OFFSET.export_kind);
+
+    if (source !== null) {
+        const withClause = deserializeOptionBoxWithClause(POS_OFFSET.with_clause);
+        return {
+            type: 'ExportNamedDeclaration',
+            start, end, declaration, specifiers, source, exportKind,
+            attributes: withClause === null ? [] : withClause.withEntries
+        };
+    }
+
+    {type: 'ExportNamedDeclaration', start, end, declaration, specifiers, source, exportKind}
+")]
 pub struct ExportNamedDeclarationConverter<'a, 'b>(pub &'b ExportNamedDeclaration<'a>);
 
 impl ESTree for ExportNamedDeclarationConverter<'_, '_> {
@@ -467,7 +602,13 @@ impl ESTree for ExportNamedDeclarationConverter<'_, '_> {
 // https://github.com/estree/estree/blob/master/es2025.md#exportnameddeclaration
 
 #[ast_meta]
-#[estree(ts_type = "Array<ImportAttribute>")]
+#[estree(
+    ts_type = "Array<ImportAttribute>",
+    raw_deser = "
+        const withClause = DESER[Option<Box<WithClause>>](POS_OFFSET.with_clause);
+        withClause === null ? [] : withClause.withEntries
+    "
+)]
 pub struct ImportDeclarationWithClause<'a, 'b>(pub &'b ImportDeclaration<'a>);
 
 impl ESTree for ImportDeclarationWithClause<'_, '_> {
@@ -481,7 +622,13 @@ impl ESTree for ImportDeclarationWithClause<'_, '_> {
 }
 
 #[ast_meta]
-#[estree(ts_type = "Array<ImportAttribute>")]
+#[estree(
+    ts_type = "Array<ImportAttribute>",
+    raw_deser = "
+        const withClause = DESER[Option<Box<WithClause>>](POS_OFFSET.with_clause);
+        withClause === null ? [] : withClause.withEntries
+    "
+)]
 pub struct ExportNamedDeclarationWithClause<'a, 'b>(pub &'b ExportNamedDeclaration<'a>);
 
 impl ESTree for ExportNamedDeclarationWithClause<'_, '_> {
@@ -495,7 +642,13 @@ impl ESTree for ExportNamedDeclarationWithClause<'_, '_> {
 }
 
 #[ast_meta]
-#[estree(ts_type = "Array<ImportAttribute>")]
+#[estree(
+    ts_type = "Array<ImportAttribute>",
+    raw_deser = "
+        const withClause = DESER[Option<Box<WithClause>>](POS_OFFSET.with_clause);
+        withClause === null ? [] : withClause.withEntries
+    "
+)]
 pub struct ExportAllDeclarationWithClause<'a, 'b>(pub &'b ExportAllDeclaration<'a>);
 
 impl ESTree for ExportAllDeclarationWithClause<'_, '_> {
@@ -516,7 +669,14 @@ impl ESTree for ExportAllDeclarationWithClause<'_, '_> {
 ///
 /// Convert to `JSXIdentifier`.
 #[ast_meta]
-#[estree(ts_type = "JSXIdentifier")]
+#[estree(
+    ts_type = "JSXIdentifier",
+    raw_deser = "
+        const ident = DESER[Box<IdentifierReference>](POS);
+        ident.type = 'JSXIdentifier';
+        ident
+    "
+)]
 pub struct JSXElementIdentifierReference<'a, 'b>(pub &'b IdentifierReference<'a>);
 
 impl ESTree for JSXElementIdentifierReference<'_, '_> {
@@ -529,7 +689,13 @@ impl ESTree for JSXElementIdentifierReference<'_, '_> {
 ///
 /// Convert to `JSXIdentifier`.
 #[ast_meta]
-#[estree(ts_type = "JSXIdentifier")]
+#[estree(
+    ts_type = "JSXIdentifier",
+    raw_deser = "
+        const thisExpr = DESER[Box<ThisExpression>](POS);
+        {type: 'JSXIdentifier', start: thisExpr.start, end: thisExpr.end, name: 'this'}
+    "
+)]
 pub struct JSXElementThisExpression<'b>(pub &'b ThisExpression);
 
 impl ESTree for JSXElementThisExpression<'_> {
@@ -537,3 +703,31 @@ impl ESTree for JSXElementThisExpression<'_> {
         JSXIdentifier { span: self.0.span, name: Atom::from("this") }.serialize(serializer);
     }
 }
+
+// --------------------
+// Comments
+// --------------------
+
+/// Serialize `value` field of `Comment`.
+///
+/// This serializer does not work for JSON serializer, because there's no access to source text
+/// in `fn serialize`. But in any case, comments often contain characters which need escaping in JSON,
+/// which is slow, so it's probably faster to transfer comments as NAPI types (which we do).
+///
+/// This meta type is only present for raw transfer, which can transfer faster.
+#[ast_meta]
+#[estree(
+    ts_type = "string",
+    raw_deser = "
+        const endCut = THIS.type === 'Line' ? 0 : 2;
+        SOURCE_TEXT.slice(THIS.start + 2, THIS.end - endCut)
+    "
+)]
+pub struct CommentValue<'b>(#[expect(dead_code)] pub &'b Comment);
+
+impl ESTree for CommentValue<'_> {
+    #[expect(clippy::unimplemented)]
+    fn serialize<S: Serializer>(&self, _serializer: S) {
+        unimplemented!();
+    }
+}

crates/oxc_ast_visit/src/generated/visit.rs

@@ -1340,6 +1340,11 @@ pub trait Visit<'a>: Sized {
     fn visit_ts_import_attribute_list(&mut self, it: &Vec<'a, TSImportAttribute<'a>>) {
         walk_ts_import_attribute_list(self, it);
     }
+
+    #[inline]
+    fn visit_spans(&mut self, it: &Vec<'a, Span>) {
+        walk_spans(self, it);
+    }
 }
 
 pub mod walk {
@@ -4412,4 +4417,11 @@ pub mod walk {
             visitor.visit_ts_import_attribute(el);
         }
     }
+
+    #[inline]
+    pub fn walk_spans<'a, V: Visit<'a>>(visitor: &mut V, it: &Vec<'a, Span>) {
+        for el in it {
+            visitor.visit_span(el);
+        }
+    }
 }

crates/oxc_ast_visit/src/generated/visit_mut.rs

@@ -1335,6 +1335,11 @@ pub trait VisitMut<'a>: Sized {
     fn visit_ts_import_attribute_list(&mut self, it: &mut Vec<'a, TSImportAttribute<'a>>) {
         walk_ts_import_attribute_list(self, it);
     }
+
+    #[inline]
+    fn visit_spans(&mut self, it: &mut Vec<'a, Span>) {
+        walk_spans(self, it);
+    }
 }
 
 pub mod walk_mut {
@@ -4657,4 +4662,11 @@ pub mod walk_mut {
             visitor.visit_ts_import_attribute(el);
         }
     }
+
+    #[inline]
+    pub fn walk_spans<'a, V: VisitMut<'a>>(visitor: &mut V, it: &mut Vec<'a, Span>) {
+        for el in it {
+            visitor.visit_span(el);
+        }
+    }
 }

crates/oxc_syntax/src/generated/assert_layouts.rs

@@ -7,13 +7,53 @@ use std::mem::{align_of, offset_of, size_of};
 
 use nonmax::NonMaxU32;
 
-use crate::{number::*, operator::*, reference::*, scope::*, symbol::*};
+use crate::{module_record::*, number::*, operator::*, reference::*, scope::*, symbol::*};
 
 #[cfg(target_pointer_width = "64")]
 const _: () = {
     assert!(size_of::<NonMaxU32>() == 4);
     assert!(align_of::<NonMaxU32>() == 4);
 
+    assert!(size_of::<NameSpan>() == 24);
+    assert!(align_of::<NameSpan>() == 8);
+    assert!(offset_of!(NameSpan, name) == 0);
+    assert!(offset_of!(NameSpan, span) == 16);
+
+    assert!(size_of::<ImportEntry>() == 96);
+    assert!(align_of::<ImportEntry>() == 8);
+    assert!(offset_of!(ImportEntry, statement_span) == 0);
+    assert!(offset_of!(ImportEntry, module_request) == 8);
+    assert!(offset_of!(ImportEntry, import_name) == 32);
+    assert!(offset_of!(ImportEntry, local_name) == 64);
+    assert!(offset_of!(ImportEntry, is_type) == 88);
+
+    assert!(size_of::<ImportImportName>() == 32);
+    assert!(align_of::<ImportImportName>() == 8);
+
+    assert!(size_of::<ExportEntry>() == 144);
+    assert!(align_of::<ExportEntry>() == 8);
+    assert!(offset_of!(ExportEntry, statement_span) == 0);
+    assert!(offset_of!(ExportEntry, span) == 8);
+    assert!(offset_of!(ExportEntry, module_request) == 16);
+    assert!(offset_of!(ExportEntry, import_name) == 40);
+    assert!(offset_of!(ExportEntry, export_name) == 72);
+    assert!(offset_of!(ExportEntry, local_name) == 104);
+    assert!(offset_of!(ExportEntry, is_type) == 136);
+
+    assert!(size_of::<ExportImportName>() == 32);
+    assert!(align_of::<ExportImportName>() == 8);
+
+    assert!(size_of::<ExportExportName>() == 32);
+    assert!(align_of::<ExportExportName>() == 8);
+
+    assert!(size_of::<ExportLocalName>() == 32);
+    assert!(align_of::<ExportLocalName>() == 8);
+
+    assert!(size_of::<DynamicImport>() == 16);
+    assert!(align_of::<DynamicImport>() == 8);
+    assert!(offset_of!(DynamicImport, span) == 0);
+    assert!(offset_of!(DynamicImport, module_request) == 8);
+
     assert!(size_of::<NumberBase>() == 1);
     assert!(align_of::<NumberBase>() == 1);
 
@@ -50,6 +90,46 @@ const _: () = {
     assert!(size_of::<NonMaxU32>() == 4);
     assert!(align_of::<NonMaxU32>() == 4);
 
+    assert!(size_of::<NameSpan>() == 16);
+    assert!(align_of::<NameSpan>() == 4);
+    assert!(offset_of!(NameSpan, name) == 0);
+    assert!(offset_of!(NameSpan, span) == 8);
+
+    assert!(size_of::<ImportEntry>() == 64);
+    assert!(align_of::<ImportEntry>() == 4);
+    assert!(offset_of!(ImportEntry, statement_span) == 0);
+    assert!(offset_of!(ImportEntry, module_request) == 8);
+    assert!(offset_of!(ImportEntry, import_name) == 24);
+    assert!(offset_of!(ImportEntry, local_name) == 44);
+    assert!(offset_of!(ImportEntry, is_type) == 60);
+
+    assert!(size_of::<ImportImportName>() == 20);
+    assert!(align_of::<ImportImportName>() == 4);
+
+    assert!(size_of::<ExportEntry>() == 96);
+    assert!(align_of::<ExportEntry>() == 4);
+    assert!(offset_of!(ExportEntry, statement_span) == 0);
+    assert!(offset_of!(ExportEntry, span) == 8);
+    assert!(offset_of!(ExportEntry, module_request) == 16);
+    assert!(offset_of!(ExportEntry, import_name) == 32);
+    assert!(offset_of!(ExportEntry, export_name) == 52);
+    assert!(offset_of!(ExportEntry, local_name) == 72);
+    assert!(offset_of!(ExportEntry, is_type) == 92);
+
+    assert!(size_of::<ExportImportName>() == 20);
+    assert!(align_of::<ExportImportName>() == 4);
+
+    assert!(size_of::<ExportExportName>() == 20);
+    assert!(align_of::<ExportExportName>() == 4);
+
+    assert!(size_of::<ExportLocalName>() == 20);
+    assert!(align_of::<ExportLocalName>() == 4);
+
+    assert!(size_of::<DynamicImport>() == 16);
+    assert!(align_of::<DynamicImport>() == 4);
+    assert!(offset_of!(DynamicImport, span) == 0);
+    assert!(offset_of!(DynamicImport, module_request) == 8);
+
     assert!(size_of::<NumberBase>() == 1);
     assert!(align_of::<NumberBase>() == 1);
 

crates/oxc_syntax/src/generated/derive_estree.rs

@@ -8,8 +8,97 @@ use oxc_estree::{
     ser::{AppendTo, AppendToConcat},
 };
 
+use crate::module_record::*;
 use crate::operator::*;
 
+impl ESTree for NameSpan<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        let mut state = serializer.serialize_struct();
+        state.serialize_field("value", &self.name);
+        state.serialize_field("start", &self.span.start);
+        state.serialize_field("end", &self.span.end);
+        state.end();
+    }
+}
+
+impl ESTree for ImportEntry<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        let mut state = serializer.serialize_struct();
+        state.serialize_field("importName", &self.import_name);
+        state.serialize_field("localName", &self.local_name);
+        state.serialize_field("isType", &self.is_type);
+        state.end();
+    }
+}
+
+impl ESTree for ImportImportName<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        match self {
+            Self::Name(it) => crate::serialize::ImportOrExportNameName(it).serialize(serializer),
+            Self::NamespaceObject => JsonSafeString("namespaceObject").serialize(serializer),
+            Self::Default(it) => {
+                crate::serialize::ImportOrExportNameDefault(it).serialize(serializer)
+            }
+        }
+    }
+}
+
+impl ESTree for ExportEntry<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        let mut state = serializer.serialize_struct();
+        state.serialize_field("start", &self.span.start);
+        state.serialize_field("end", &self.span.end);
+        state.serialize_field("moduleRequest", &self.module_request);
+        state.serialize_field("importName", &self.import_name);
+        state.serialize_field("exportName", &self.export_name);
+        state.serialize_field("localName", &self.local_name);
+        state.end();
+    }
+}
+
+impl ESTree for ExportImportName<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        match self {
+            Self::Name(it) => crate::serialize::ImportOrExportNameName(it).serialize(serializer),
+            Self::All => JsonSafeString("all").serialize(serializer),
+            Self::AllButDefault => JsonSafeString("allButDefault").serialize(serializer),
+            Self::Null => JsonSafeString("null").serialize(serializer),
+        }
+    }
+}
+
+impl ESTree for ExportExportName<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        match self {
+            Self::Name(it) => crate::serialize::ImportOrExportNameName(it).serialize(serializer),
+            Self::Default(it) => {
+                crate::serialize::ImportOrExportNameDefault(it).serialize(serializer)
+            }
+            Self::Null => JsonSafeString("null").serialize(serializer),
+        }
+    }
+}
+
+impl ESTree for ExportLocalName<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        match self {
+            Self::Name(it) => crate::serialize::ImportOrExportNameName(it).serialize(serializer),
+            Self::Default(it) => crate::serialize::ExportLocalNameDefault(it).serialize(serializer),
+            Self::Null => JsonSafeString("null").serialize(serializer),
+        }
+    }
+}
+
+impl ESTree for DynamicImport {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        let mut state = serializer.serialize_struct();
+        state.serialize_field("start", &self.span.start);
+        state.serialize_field("end", &self.span.end);
+        state.serialize_field("moduleRequest", &self.module_request);
+        state.end();
+    }
+}
+
 impl ESTree for AssignmentOperator {
     fn serialize<S: Serializer>(&self, serializer: S) {
         match self {

crates/oxc_syntax/src/lib.rs

@@ -16,6 +16,8 @@ pub mod operator;
 pub mod precedence;
 pub mod reference;
 pub mod scope;
+#[cfg(feature = "serialize")]
+mod serialize;
 pub mod symbol;
 pub mod xml_entities;
 

crates/oxc_syntax/src/module_record.rs

@@ -3,6 +3,8 @@
 use rustc_hash::FxHashMap;
 
 use oxc_allocator::{Allocator, Vec};
+use oxc_ast_macros::ast;
+use oxc_estree::ESTree;
 use oxc_span::{Atom, Span};
 
 /// ESM Module Record
@@ -82,9 +84,13 @@ impl<'a> ModuleRecord<'a> {
 }
 
 /// Name and Span
+#[ast]
 #[derive(Debug, Clone, PartialEq, Eq)]
+#[generate_derive(ESTree)]
+#[estree(no_type, no_ts_def)]
 pub struct NameSpan<'a> {
     /// Name
+    #[estree(rename = "value")]
     pub name: Atom<'a>,
 
     /// Span
@@ -113,9 +119,13 @@ impl<'a> NameSpan<'a> {
 ///
 /// import * as ns from "mod";
 /// ```
+#[ast]
 #[derive(Debug, Clone, PartialEq, Eq)]
+#[generate_derive(ESTree)]
+#[estree(no_type, no_ts_def)]
 pub struct ImportEntry<'a> {
     /// Span of the import statement.
+    #[estree(skip)]
     pub statement_span: Span,
 
     /// String value of the ModuleSpecifier of the ImportDeclaration.
@@ -126,6 +136,7 @@ pub struct ImportEntry<'a> {
     /// import { foo } from "mod";
     /// //                   ^^^
     /// ```
+    #[estree(skip)]
     pub module_request: NameSpan<'a>,
 
     /// The name under which the desired binding is exported by the module identified by `[[ModuleRequest]]`.
@@ -172,14 +183,20 @@ pub struct ImportEntry<'a> {
 }
 
 /// `ImportName` For `ImportEntry`
+#[ast]
 #[derive(Debug, Clone, PartialEq, Eq)]
+#[generate_derive(ESTree)]
+#[estree(no_ts_def)]
 pub enum ImportImportName<'a> {
     /// `import { x } from "mod"`
-    Name(NameSpan<'a>),
+    #[estree(via = ImportOrExportNameName)]
+    Name(NameSpan<'a>) = 0,
     /// `import * as ns from "mod"`
-    NamespaceObject,
+    #[estree(via = ImportImportNameNamespaceObject)]
+    NamespaceObject = 1,
     /// `import defaultExport from "mod"`
-    Default(Span),
+    #[estree(via = ImportOrExportNameDefault)]
+    Default(Span) = 2,
 }
 
 impl ImportImportName<'_> {
@@ -213,9 +230,13 @@ impl ImportImportName<'_> {
 /// //              ^^^ export_name
 ///
 /// ```
+#[ast]
 #[derive(Debug, Default, Clone, PartialEq, Eq)]
+#[generate_derive(ESTree)]
+#[estree(no_type, no_ts_def)]
 pub struct ExportEntry<'a> {
     /// Span of the import statement.
+    #[estree(skip)]
     pub statement_span: Span,
 
     /// Span for the entire export entry
@@ -249,21 +270,29 @@ pub struct ExportEntry<'a> {
     /// export { type foo }
     /// export type { foo } from 'mod'
     /// ```
+    #[estree(skip)]
     pub is_type: bool,
 }
 
 /// `ImportName` for `ExportEntry`
+#[ast]
 #[derive(Debug, Default, Clone, PartialEq, Eq)]
+#[generate_derive(ESTree)]
+#[estree(no_ts_def)]
 pub enum ExportImportName<'a> {
     /// Name
-    Name(NameSpan<'a>),
+    #[estree(via = ImportOrExportNameName)]
+    Name(NameSpan<'a>) = 0,
     /// all is used for export * as ns from "mod" declarations.
-    All,
+    #[estree(via = ExportImportNameAll)]
+    All = 1,
     /// all-but-default is used for export * from "mod" declarations.
-    AllButDefault,
+    #[estree(via = ExportImportNameAllButDefault)]
+    AllButDefault = 2,
     /// the ExportDeclaration does not have a ModuleSpecifier
     #[default]
-    Null,
+    #[estree(via = ExportNameNull)]
+    Null = 3,
 }
 
 /// Export Import Name
@@ -280,15 +309,21 @@ impl ExportImportName<'_> {
 }
 
 /// `ExportName` for `ExportEntry`
+#[ast]
 #[derive(Debug, Default, Clone, PartialEq, Eq)]
+#[generate_derive(ESTree)]
+#[estree(no_ts_def)]
 pub enum ExportExportName<'a> {
     /// Name
-    Name(NameSpan<'a>),
+    #[estree(via = ImportOrExportNameName)]
+    Name(NameSpan<'a>) = 0,
     /// Default
-    Default(Span),
+    #[estree(via = ImportOrExportNameDefault)]
+    Default(Span) = 1,
     /// Null
+    #[estree(via = ExportNameNull)]
     #[default]
-    Null,
+    Null = 2,
 }
 
 impl ExportExportName<'_> {
@@ -324,15 +359,21 @@ impl ExportExportName<'_> {
 }
 
 /// `LocalName` for `ExportEntry`
+#[ast]
 #[derive(Debug, Default, Clone, PartialEq, Eq)]
+#[generate_derive(ESTree)]
+#[estree(no_ts_def)]
 pub enum ExportLocalName<'a> {
     /// Name
-    Name(NameSpan<'a>),
+    #[estree(via = ImportOrExportNameName)]
+    Name(NameSpan<'a>) = 0,
     /// `export default name_span`
-    Default(NameSpan<'a>),
+    #[estree(via = ExportLocalNameDefault)]
+    Default(NameSpan<'a>) = 1,
     /// Null
     #[default]
-    Null,
+    #[estree(via = ExportNameNull)]
+    Null = 2,
 }
 
 impl<'a> ExportLocalName<'a> {
@@ -379,11 +420,15 @@ pub struct RequestedModule {
 }
 
 /// Dynamic import expression.
+#[ast]
 #[derive(Debug, Clone, Copy)]
+#[generate_derive(ESTree)]
+#[estree(no_type, no_ts_def)]
 pub struct DynamicImport {
     /// Span of the import expression.
     pub span: Span,
     /// Span the ModuleSpecifier, which is an expression.
+    #[estree(no_flatten)]
     pub module_request: Span,
 }
 

crates/oxc_syntax/src/serialize.rs

@@ -0,0 +1,93 @@
+use oxc_ast_macros::ast_meta;
+use oxc_estree::{ESTree, Serializer};
+use oxc_span::Span;
+
+use crate::module_record::NameSpan;
+
+/// Macro to create a dummy `ESTree` impl for a type.
+///
+/// The meta types in this module are only used for raw transfer,
+/// so no need for real `ESTree` impls.
+macro_rules! dummy_estree_impl {
+    ($ty:ty) => {
+        impl ESTree for $ty {
+            fn serialize<S: Serializer>(&self, _serializer: S) {
+                unimplemented!();
+            }
+        }
+    };
+}
+
+/// Serializer for `Name` variant of `ExportLocalName`, `ExportExportName`, `ExportImportName`,
+/// and `ImportImportName`.
+#[ast_meta]
+#[estree(
+    ts_type = "Dummy",
+    raw_deser = "
+        var nameSpan = DESER[NameSpan](POS);
+        {kind: 'Name', name: nameSpan.value, start: nameSpan.start, end: nameSpan.end}
+    "
+)]
+pub struct ImportOrExportNameName<'a, 'b>(#[expect(dead_code)] pub &'b NameSpan<'a>);
+
+dummy_estree_impl!(ImportOrExportNameName<'_, '_>);
+
+/// Serializer for `Default` variant of `ExportLocalName`.
+#[ast_meta]
+#[estree(
+    ts_type = "Dummy",
+    raw_deser = "
+        var nameSpan = DESER[NameSpan](POS);
+        {kind: 'Default', name: nameSpan.value, start: nameSpan.start, end: nameSpan.end}
+    "
+)]
+pub struct ExportLocalNameDefault<'a, 'b>(#[expect(dead_code)] pub &'b NameSpan<'a>);
+
+dummy_estree_impl!(ExportLocalNameDefault<'_, '_>);
+
+/// Serializer for `Null` variant of `ExportLocalName`, `ExportExportName`, and `ExportImportName`.
+#[ast_meta]
+#[estree(ts_type = "Dummy", raw_deser = "{kind: 'None', name: null, start: null, end: null}")]
+pub struct ExportNameNull;
+
+dummy_estree_impl!(ExportNameNull);
+
+/// Serializer for `Default` variant of `ExportExportName` and `ImportImportName`.
+#[ast_meta]
+#[estree(
+    ts_type = "Dummy",
+    raw_deser = "
+        var span = DESER[Span](POS);
+        {kind: 'Default', name: null, start: span.start, end: span.end}
+    "
+)]
+pub struct ImportOrExportNameDefault<'b>(#[expect(dead_code)] pub &'b Span);
+
+dummy_estree_impl!(ImportOrExportNameDefault<'_>);
+
+/// Serializer for `All` variant of `ExportImportName`.
+#[ast_meta]
+#[estree(ts_type = "Dummy", raw_deser = "{kind: 'All', name: null, start: null, end: null}")]
+pub struct ExportImportNameAll;
+
+dummy_estree_impl!(ExportImportNameAll);
+
+/// Serializer for `AllButDefault` variant of `ExportImportName`.
+#[ast_meta]
+#[estree(
+    ts_type = "Dummy",
+    raw_deser = "{kind: 'AllButDefault', name: null, start: null, end: null}"
+)]
+pub struct ExportImportNameAllButDefault;
+
+dummy_estree_impl!(ExportImportNameAllButDefault);
+
+/// Serializer for `NamespaceObject` variant of `ImportImportName`.
+#[ast_meta]
+#[estree(
+    ts_type = "Dummy",
+    raw_deser = "{kind: 'NamespaceObject', name: null, start: null, end: null}"
+)]
+pub struct ImportImportNameNamespaceObject;
+
+dummy_estree_impl!(ImportImportNameNamespaceObject);

napi/parser/Cargo.toml

@@ -23,6 +23,8 @@ doctest = false
 
 [dependencies]
 oxc = { workspace = true, features = ["ast_visit", "semantic", "serialize"] }
+oxc_ast_macros = { workspace = true }
+oxc_estree = { workspace = true }
 oxc_napi = { workspace = true }
 
 rustc-hash = { workspace = true }

napi/parser/bench.bench.mjs

@@ -36,6 +36,14 @@ const fixtures = await Promise.all(fixtureUrls.map(async (url) => {
 // Run benchmarks
 for (const { filename, code } of fixtures) {
   bench(`parser_napi[${filename}]`, () => {
-    parseSync(filename, code);
+    const ret = parseSync(filename, code);
+    // Read returned object's properties to execute getters which deserialize
+    const { program, comments, module, errors } = ret;
+  });
+
+  bench(`parser_napi_raw[${filename}]`, () => {
+    const ret = parseSync(filename, code, { experimentalRawTransfer: true });
+    // Read returned object's properties to execute getters
+    const { program, comments, module, errors } = ret;
   });
 }

napi/parser/bindings.js

@@ -374,7 +374,9 @@ module.exports.ParseResult = nativeBinding.ParseResult
 module.exports.ExportExportNameKind = nativeBinding.ExportExportNameKind
 module.exports.ExportImportNameKind = nativeBinding.ExportImportNameKind
 module.exports.ExportLocalNameKind = nativeBinding.ExportLocalNameKind
+module.exports.getBufferOffset = nativeBinding.getBufferOffset
 module.exports.ImportNameKind = nativeBinding.ImportNameKind
 module.exports.parseAsync = nativeBinding.parseAsync
 module.exports.parseSync = nativeBinding.parseSync
+module.exports.parseSyncRaw = nativeBinding.parseSyncRaw
 module.exports.Severity = nativeBinding.Severity

napi/parser/index.d.ts

@@ -100,6 +100,14 @@ export declare const enum ExportLocalNameKind {
   None = 'None'
 }
 
+/**
+ * Get offset within a `Uint8Array` which is aligned on 4 GiB.
+ *
+ * Does not check that the offset is within bounds of `buffer`.
+ * To ensure it always is, provide a `Uint8Array` of at least 4 GiB size.
+ */
+export declare function getBufferOffset(buffer: Uint8Array): number
+
 export interface ImportName {
   kind: ImportNameKind
   name?: string
@@ -157,6 +165,34 @@ export interface ParserOptions {
 /** Parse synchronously. */
 export declare function parseSync(filename: string, sourceText: string, options?: ParserOptions | undefined | null): ParseResult
 
+/**
+ * Parses AST into provided `Uint8Array` buffer.
+ *
+ * Source text must be written into the start of the buffer, and its length (in UTF-8 bytes)
+ * provided as `source_len`.
+ *
+ * This function will parse the source, and write the AST into the buffer, starting at the end.
+ *
+ * It also writes to the very end of the buffer the offset of `Program` within the buffer.
+ *
+ * Caller can deserialize data from the buffer on JS side.
+ *
+ * # SAFETY
+ *
+ * Caller must ensure:
+ * * Source text is written into start of the buffer.
+ * * Source text's UTF-8 byte length is `source_len`.
+ * * The 1st `source_len` bytes of the buffer comprises a valid UTF-8 string.
+ *
+ * If source text is originally a JS string on JS side, and converted to a buffer with
+ * `Buffer.from(str)` or `new TextEncoder().encode(str)`, this guarantees it's valid UTF-8.
+ *
+ * # Panics
+ *
+ * Panics if source text is too long, or AST takes more memory than is available in the buffer.
+ */
+export declare function parseSyncRaw(filename: string, buffer: Uint8Array, sourceLen: number, options?: ParserOptions | undefined | null): void
+
 export declare const enum Severity {
   Error = 'Error',
   Warning = 'Warning',

napi/parser/index.js

@@ -1,4 +1,5 @@
 const bindings = require('./bindings.js');
+const deserialize = require('./deserialize.js');
 
 module.exports.ParseResult = bindings.ParseResult;
 module.exports.ExportExportNameKind = bindings.ExportExportNameKind;
@@ -55,6 +56,77 @@ module.exports.parseAsync = async function parseAsync(...args) {
   return wrap(await bindings.parseAsync(...args));
 };
 
-module.exports.parseSync = function parseSync(...args) {
-  return wrap(bindings.parseSync(...args));
+module.exports.parseSync = function parseSync(filename, sourceText, options) {
+  if (options?.experimentalRawTransfer) {
+    return parseSyncRaw(filename, sourceText, options);
+  }
+
+  return wrap(bindings.parseSync(filename, sourceText, options));
 };
+
+let buffer, encoder;
+
+function parseSyncRaw(filename, sourceText, options) {
+  // Delete `experimentalRawTransfer` option
+  let experimentalRawTransfer;
+  ({ experimentalRawTransfer, ...options } = options);
+
+  // Create buffer and `TextEncoder`
+  if (!buffer) {
+    buffer = createBuffer();
+    encoder = new TextEncoder();
+  }
+
+  // Write source into start of buffer.
+  // `TextEncoder` cannot write into a `Uint8Array` larger than 1 GiB,
+  // so create a view into buffer of this size to write into.
+  const sourceBuffer = new Uint8Array(buffer.buffer, buffer.byteOffset, ONE_GIB);
+  const { read, written: sourceByteLen } = encoder.encodeInto(sourceText, sourceBuffer);
+  if (read !== sourceText.length) {
+    throw new Error('Failed to write source text into buffer');
+  }
+
+  // Parse
+  bindings.parseSyncRaw(filename, buffer, sourceByteLen, options);
+
+  // Deserialize.
+  // We cannot lazily deserialize in the getters, because the buffer might be re-used to parse
+  // another file before the getter is called.
+  const data = deserialize(buffer, sourceText, sourceByteLen);
+
+  return {
+    get program() {
+      return data.program;
+    },
+    get module() {
+      return data.module;
+    },
+    get comments() {
+      return data.comments;
+    },
+    get errors() {
+      return data.errors;
+    },
+  };
+}
+
+const ONE_GIB = 1 << 30,
+  TWO_GIB = ONE_GIB * 2,
+  SIX_GIB = ONE_GIB * 6;
+
+// Create a `Uint8Array` which is 2 GiB in size, with its start aligned on 4 GiB.
+//
+// Achieve this by creating a 6 GiB `ArrayBuffer`, getting the offset within it that's aligned to 4 GiB,
+// chopping off that number of bytes from the start, and shortening to 2 GiB.
+//
+// It's always possible to obtain a 2 GiB slice aligned on 4 GiB within a 6 GiB buffer,
+// no matter how the 6 GiB buffer is aligned.
+//
+// Note: On systems with virtual memory, this only consumes 6 GiB of *virtual* memory.
+// It does not consume physical memory until data is actually written to the `Uint8Array`.
+// Physical memory consumed corresponds to the quantity of data actually written.
+function createBuffer() {
+  const arrayBuffer = new ArrayBuffer(SIX_GIB);
+  const offset = bindings.getBufferOffset(new Uint8Array(arrayBuffer));
+  return new Uint8Array(arrayBuffer, offset, TWO_GIB);
+}

napi/parser/src/generated/assert_layouts.rs

@@ -0,0 +1,99 @@
+// Auto-generated code, DO NOT EDIT DIRECTLY!
+// To edit this generated file you have to edit `tasks/ast_tools/src/generators/assert_layouts.rs`
+
+#![allow(unused_imports)]
+
+use std::mem::{align_of, offset_of, size_of};
+
+use crate::raw_transfer_types::*;
+
+#[cfg(target_pointer_width = "64")]
+const _: () = {
+    assert!(size_of::<RawTransferData>() == 360);
+    assert!(align_of::<RawTransferData>() == 8);
+    assert!(offset_of!(RawTransferData, program) == 0);
+    assert!(offset_of!(RawTransferData, comments) == 160);
+    assert!(offset_of!(RawTransferData, module) == 192);
+    assert!(offset_of!(RawTransferData, errors) == 328);
+
+    assert!(size_of::<Error>() == 72);
+    assert!(align_of::<Error>() == 8);
+    assert!(offset_of!(Error, severity) == 0);
+    assert!(offset_of!(Error, message) == 8);
+    assert!(offset_of!(Error, labels) == 24);
+    assert!(offset_of!(Error, help_message) == 56);
+
+    assert!(size_of::<ErrorSeverity>() == 1);
+    assert!(align_of::<ErrorSeverity>() == 1);
+
+    assert!(size_of::<ErrorLabel>() == 24);
+    assert!(align_of::<ErrorLabel>() == 8);
+    assert!(offset_of!(ErrorLabel, message) == 0);
+    assert!(offset_of!(ErrorLabel, span) == 16);
+
+    assert!(size_of::<EcmaScriptModule>() == 136);
+    assert!(align_of::<EcmaScriptModule>() == 8);
+    assert!(offset_of!(EcmaScriptModule, has_module_syntax) == 0);
+    assert!(offset_of!(EcmaScriptModule, static_imports) == 8);
+    assert!(offset_of!(EcmaScriptModule, static_exports) == 40);
+    assert!(offset_of!(EcmaScriptModule, dynamic_imports) == 72);
+    assert!(offset_of!(EcmaScriptModule, import_metas) == 104);
+
+    assert!(size_of::<StaticImport>() == 64);
+    assert!(align_of::<StaticImport>() == 8);
+    assert!(offset_of!(StaticImport, span) == 0);
+    assert!(offset_of!(StaticImport, module_request) == 8);
+    assert!(offset_of!(StaticImport, entries) == 32);
+
+    assert!(size_of::<StaticExport>() == 40);
+    assert!(align_of::<StaticExport>() == 8);
+    assert!(offset_of!(StaticExport, span) == 0);
+    assert!(offset_of!(StaticExport, entries) == 8);
+};
+
+#[cfg(target_pointer_width = "32")]
+const _: () = {
+    assert!(size_of::<RawTransferData>() == 188);
+    assert!(align_of::<RawTransferData>() == 4);
+    assert!(offset_of!(RawTransferData, program) == 0);
+    assert!(offset_of!(RawTransferData, comments) == 88);
+    assert!(offset_of!(RawTransferData, module) == 104);
+    assert!(offset_of!(RawTransferData, errors) == 172);
+
+    assert!(size_of::<Error>() == 36);
+    assert!(align_of::<Error>() == 4);
+    assert!(offset_of!(Error, severity) == 0);
+    assert!(offset_of!(Error, message) == 4);
+    assert!(offset_of!(Error, labels) == 12);
+    assert!(offset_of!(Error, help_message) == 28);
+
+    assert!(size_of::<ErrorSeverity>() == 1);
+    assert!(align_of::<ErrorSeverity>() == 1);
+
+    assert!(size_of::<ErrorLabel>() == 16);
+    assert!(align_of::<ErrorLabel>() == 4);
+    assert!(offset_of!(ErrorLabel, message) == 0);
+    assert!(offset_of!(ErrorLabel, span) == 8);
+
+    assert!(size_of::<EcmaScriptModule>() == 68);
+    assert!(align_of::<EcmaScriptModule>() == 4);
+    assert!(offset_of!(EcmaScriptModule, has_module_syntax) == 0);
+    assert!(offset_of!(EcmaScriptModule, static_imports) == 4);
+    assert!(offset_of!(EcmaScriptModule, static_exports) == 20);
+    assert!(offset_of!(EcmaScriptModule, dynamic_imports) == 36);
+    assert!(offset_of!(EcmaScriptModule, import_metas) == 52);
+
+    assert!(size_of::<StaticImport>() == 40);
+    assert!(align_of::<StaticImport>() == 4);
+    assert!(offset_of!(StaticImport, span) == 0);
+    assert!(offset_of!(StaticImport, module_request) == 8);
+    assert!(offset_of!(StaticImport, entries) == 24);
+
+    assert!(size_of::<StaticExport>() == 24);
+    assert!(align_of::<StaticExport>() == 4);
+    assert!(offset_of!(StaticExport, span) == 0);
+    assert!(offset_of!(StaticExport, entries) == 8);
+};
+
+#[cfg(not(any(target_pointer_width = "64", target_pointer_width = "32")))]
+const _: () = panic!("Platforms with pointer width other than 64 or 32 bit are not supported");

napi/parser/src/generated/derive_estree.rs

@@ -0,0 +1,86 @@
+// Auto-generated code, DO NOT EDIT DIRECTLY!
+// To edit this generated file you have to edit `tasks/ast_tools/src/derives/estree.rs`
+
+#![allow(unused_imports, clippy::match_same_arms, clippy::semicolon_if_nothing_returned)]
+
+use oxc_estree::{
+    ESTree, FlatStructSerializer, JsonSafeString, Serializer, StructSerializer,
+    ser::{AppendTo, AppendToConcat},
+};
+
+use crate::raw_transfer_types::*;
+
+impl ESTree for RawTransferData<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        let mut state = serializer.serialize_struct();
+        state.serialize_field("program", &self.program);
+        state.serialize_field("comments", &self.comments);
+        state.serialize_field("module", &self.module);
+        state.serialize_field("errors", &self.errors);
+        state.end();
+    }
+}
+
+impl ESTree for Error<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        let mut state = serializer.serialize_struct();
+        state.serialize_field("severity", &self.severity);
+        state.serialize_field("message", &self.message);
+        state.serialize_field("labels", &self.labels);
+        state.serialize_field("helpMessage", &self.help_message);
+        state.end();
+    }
+}
+
+impl ESTree for ErrorSeverity {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        match self {
+            Self::Error => JsonSafeString("Error").serialize(serializer),
+            Self::Warning => JsonSafeString("Warning").serialize(serializer),
+            Self::Advice => JsonSafeString("Advice").serialize(serializer),
+        }
+    }
+}
+
+impl ESTree for ErrorLabel<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        let mut state = serializer.serialize_struct();
+        state.serialize_field("message", &self.message);
+        state.serialize_field("start", &self.span.start);
+        state.serialize_field("end", &self.span.end);
+        state.end();
+    }
+}
+
+impl ESTree for EcmaScriptModule<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        let mut state = serializer.serialize_struct();
+        state.serialize_field("hasModuleSyntax", &self.has_module_syntax);
+        state.serialize_field("staticImports", &self.static_imports);
+        state.serialize_field("staticExports", &self.static_exports);
+        state.serialize_field("dynamicImports", &self.dynamic_imports);
+        state.serialize_field("importMetas", &self.import_metas);
+        state.end();
+    }
+}
+
+impl ESTree for StaticImport<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        let mut state = serializer.serialize_struct();
+        state.serialize_field("start", &self.span.start);
+        state.serialize_field("end", &self.span.end);
+        state.serialize_field("moduleRequest", &self.module_request);
+        state.serialize_field("entries", &self.entries);
+        state.end();
+    }
+}
+
+impl ESTree for StaticExport<'_> {
+    fn serialize<S: Serializer>(&self, serializer: S) {
+        let mut state = serializer.serialize_struct();
+        state.serialize_field("start", &self.span.start);
+        state.serialize_field("end", &self.span.end);
+        state.serialize_field("entries", &self.entries);
+        state.end();
+    }
+}

napi/parser/src/lib.rs

@@ -17,9 +17,18 @@ use oxc::{
 use oxc_napi::OxcError;
 
 mod convert;
+mod raw_transfer;
+mod raw_transfer_types;
 mod types;
+pub use raw_transfer::{get_buffer_offset, parse_sync_raw};
 pub use types::{Comment, EcmaScriptModule, ParseResult, ParserOptions};
 
+mod generated {
+    // Note: We intentionally don't import `generated/derive_estree.rs`. It's not needed.
+    #[cfg(debug_assertions)]
+    pub mod assert_layouts;
+}
+
 fn get_source_type(filename: &str, options: &ParserOptions) -> SourceType {
     match options.lang.as_deref() {
         Some("js") => SourceType::mjs(),

napi/parser/src/raw_transfer.rs

@@ -0,0 +1,174 @@
+use std::{
+    mem::{self, ManuallyDrop},
+    ptr::{self, NonNull},
+    str,
+};
+
+use napi::bindgen_prelude::Uint8Array;
+use napi_derive::napi;
+
+use oxc::{
+    allocator::{Allocator, FromIn, Vec as ArenaVec},
+    ast_visit::utf8_to_utf16::Utf8ToUtf16,
+};
+
+use crate::{
+    ParserOptions, get_source_type, parse,
+    raw_transfer_types::{EcmaScriptModule, Error, RawTransferData},
+};
+
+// For raw transfer, use a buffer 4 GiB in size, with 4 GiB alignment.
+// This ensures that all 64-bit pointers have the same value in upper 32 bits,
+// so JS only needs to read the lower 32 bits to get an offset into the buffer.
+// However, only use first half of buffer (2 GiB) for the arena, so 32-bit offsets
+// don't have the highest bit set. JS bitwise operators interpret the highest bit as sign bit,
+// so this enables using `>>` bitshift operator in JS, rather than the more expensive `>>>`,
+// without offsets being interpreted as negative.
+const TWO_GIB: usize = 1 << 31;
+const FOUR_GIB: usize = 1 << 32;
+
+const BUFFER_SIZE: usize = TWO_GIB;
+const BUFFER_ALIGN: usize = FOUR_GIB;
+const BUMP_ALIGN: usize = 16;
+
+/// Get offset within a `Uint8Array` which is aligned on 4 GiB.
+///
+/// Does not check that the offset is within bounds of `buffer`.
+/// To ensure it always is, provide a `Uint8Array` of at least 4 GiB size.
+#[napi]
+pub fn get_buffer_offset(buffer: Uint8Array) -> u32 {
+    let buffer = &*buffer;
+    let buffer_addr32 = buffer.as_ptr() as u32;
+    0u32.wrapping_sub(buffer_addr32)
+}
+
+/// Parses AST into provided `Uint8Array` buffer.
+///
+/// Source text must be written into the start of the buffer, and its length (in UTF-8 bytes)
+/// provided as `source_len`.
+///
+/// This function will parse the source, and write the AST into the buffer, starting at the end.
+///
+/// It also writes to the very end of the buffer the offset of `Program` within the buffer.
+///
+/// Caller can deserialize data from the buffer on JS side.
+///
+/// # SAFETY
+///
+/// Caller must ensure:
+/// * Source text is written into start of the buffer.
+/// * Source text's UTF-8 byte length is `source_len`.
+/// * The 1st `source_len` bytes of the buffer comprises a valid UTF-8 string.
+///
+/// If source text is originally a JS string on JS side, and converted to a buffer with
+/// `Buffer.from(str)` or `new TextEncoder().encode(str)`, this guarantees it's valid UTF-8.
+///
+/// # Panics
+///
+/// Panics if source text is too long, or AST takes more memory than is available in the buffer.
+#[allow(clippy::items_after_statements, clippy::allow_attributes)]
+#[napi]
+pub unsafe fn parse_sync_raw(
+    filename: String,
+    mut buffer: Uint8Array,
+    source_len: u32,
+    options: Option<ParserOptions>,
+) {
+    // 32-bit systems are not supported
+    const { assert!(std::mem::size_of::<usize>() >= 8) };
+
+    // Check buffer has expected size and alignment
+    let buffer = &mut *buffer;
+    assert_eq!(buffer.len(), BUFFER_SIZE);
+    let buffer_ptr = ptr::from_mut(buffer).cast::<u8>();
+    assert!(is_multiple_of(buffer_ptr as usize, BUFFER_ALIGN));
+
+    // Get offsets and size of data region to be managed by arena allocator.
+    // Leave space for source before it, and 16 bytes for metadata after it.
+    // Metadata actually only takes 4 bytes, but round everything up to multiple of 16,
+    // as `bumpalo` requires that alignment.
+    const METADATA_SIZE: usize = 16;
+    const {
+        assert!(METADATA_SIZE >= BUMP_ALIGN);
+        assert!(is_multiple_of(METADATA_SIZE, BUMP_ALIGN));
+    };
+    let source_len = source_len as usize;
+    let data_offset = source_len.next_multiple_of(BUMP_ALIGN);
+    let data_size = BUFFER_SIZE.saturating_sub(data_offset + METADATA_SIZE);
+    assert!(data_size >= Allocator::RAW_MIN_SIZE, "Source text is too long");
+
+    // Create `Allocator`.
+    // Wrap in `ManuallyDrop` so the allocation doesn't get freed at end of function, or if panic.
+    // SAFETY: `data_offset` is less than `buffer.len()`, so `.add(data_offset)` cannot wrap
+    // or be out of bounds.
+    let data_ptr = unsafe { buffer_ptr.add(data_offset) };
+    debug_assert!(is_multiple_of(data_ptr as usize, BUMP_ALIGN));
+    debug_assert!(is_multiple_of(data_size, BUMP_ALIGN));
+    // SAFETY: `data_ptr` and `data_size` outline a section of the memory in `buffer`.
+    // `data_ptr` and `data_size` are multiples of 16.
+    // `data_size` is greater than `Allocator::MIN_SIZE`.
+    let allocator =
+        unsafe { Allocator::from_raw_parts(NonNull::new_unchecked(data_ptr), data_size) };
+    let allocator = ManuallyDrop::new(allocator);
+
+    // Parse source.
+    // Enclose parsing logic in a scope to make 100% sure no references to within `Allocator`
+    // exist after this.
+    let options = options.unwrap_or_default();
+    let source_type = get_source_type(&filename, &options);
+
+    let data_ptr = {
+        // SAFETY: We checked above that `source_len` does not exceed length of buffer
+        let source_text = unsafe { buffer.get_unchecked(..source_len) };
+        // SAFETY: Caller guarantees source occupies this region of the buffer and is valid UTF-8
+        let source_text = unsafe { str::from_utf8_unchecked(source_text) };
+
+        let ret = parse(&allocator, source_type, source_text, &options);
+        let mut program = ret.program;
+        let mut comments = mem::replace(&mut program.comments, ArenaVec::new_in(&allocator));
+        let mut module_record = ret.module_record;
+
+        // Convert errors
+        let mut errors = ArenaVec::from_iter_in(
+            ret.errors.iter().map(|error| Error::from_in(error, &allocator)),
+            &allocator,
+        );
+
+        // Convert spans to UTF-16
+        let span_converter = Utf8ToUtf16::new(source_text);
+        span_converter.convert_program(&mut program);
+        span_converter.convert_comments(&mut comments);
+        span_converter.convert_module_record(&mut module_record);
+        if let Some(mut converter) = span_converter.converter() {
+            for error in &mut errors {
+                for label in &mut error.labels {
+                    converter.convert_span(&mut label.span);
+                }
+            }
+        }
+
+        // Convert module record
+        let module = EcmaScriptModule::from_in(module_record, &allocator);
+
+        let data = RawTransferData { program, comments, module, errors };
+        let data = allocator.alloc(data);
+
+        // Return pointer to `RawTransferData`
+        ptr::from_ref(data).cast::<u8>()
+    };
+
+    // Write offset of `RawTransferData` into end of buffer
+    #[allow(clippy::cast_possible_truncation)]
+    let data_offset = data_ptr as u32;
+    const METADATA_OFFSET: usize = BUFFER_SIZE - METADATA_SIZE;
+    // SAFETY: `METADATA_OFFSET` is less than length of `buffer`
+    #[expect(clippy::cast_ptr_alignment)]
+    unsafe {
+        buffer_ptr.add(METADATA_OFFSET).cast::<u32>().write(data_offset);
+    }
+}
+
+/// Returns `true` if `n` is a multiple of `divisor`.
+const fn is_multiple_of(n: usize, divisor: usize) -> bool {
+    n % divisor == 0
+}

napi/parser/src/raw_transfer_types.rs

@@ -0,0 +1,202 @@
+use rustc_hash::FxHashMap;
+
+use oxc::{
+    allocator::{Allocator, FromIn, Vec},
+    ast::ast::{Comment, Program},
+    diagnostics::{LabeledSpan, OxcDiagnostic, Severity},
+    span::{Atom, Span},
+    syntax::module_record::{DynamicImport, ExportEntry, ImportEntry, ModuleRecord, NameSpan},
+};
+use oxc_ast_macros::ast;
+use oxc_estree::ESTree;
+
+/// The main struct containing all deserializable data in raw transfer.
+#[ast]
+#[generate_derive(ESTree)]
+#[estree(no_type, no_ts_def)]
+pub struct RawTransferData<'a> {
+    pub program: Program<'a>,
+    pub comments: Vec<'a, Comment>,
+    pub module: EcmaScriptModule<'a>,
+    pub errors: Vec<'a, Error<'a>>,
+}
+
+// Errors.
+//
+// These types and the `From` / `FromIn` impls mirror the implementation in `types.rs`
+// and `crates/oxc_napi/src/lib.rs`.
+// Only difference is that these versions of the types are arena-allocated.
+
+#[ast]
+#[generate_derive(ESTree)]
+#[estree(no_type, no_ts_def)]
+pub struct Error<'a> {
+    pub severity: ErrorSeverity,
+    pub message: Atom<'a>,
+    pub labels: Vec<'a, ErrorLabel<'a>>,
+    pub help_message: Option<Atom<'a>>,
+}
+
+impl<'a> FromIn<'a, &OxcDiagnostic> for Error<'a> {
+    fn from_in(diagnostic: &OxcDiagnostic, allocator: &'a Allocator) -> Self {
+        let labels = diagnostic.labels.as_ref().map_or_else(
+            || Vec::new_in(allocator),
+            |labels| {
+                Vec::from_iter_in(
+                    labels.iter().map(|label| ErrorLabel::from_in(label, allocator)),
+                    allocator,
+                )
+            },
+        );
+
+        Self {
+            severity: ErrorSeverity::from(diagnostic.severity),
+            message: Atom::from_in(diagnostic.message.as_ref(), allocator),
+            labels,
+            help_message: diagnostic
+                .help
+                .as_ref()
+                .map(|help| Atom::from_in(help.as_ref(), allocator)),
+        }
+    }
+}
+
+#[ast]
+#[derive(Clone, Copy)]
+#[generate_derive(ESTree)]
+#[estree(no_rename_variants, no_ts_def)]
+pub enum ErrorSeverity {
+    Error = 0,
+    Warning = 1,
+    Advice = 2,
+}
+
+impl From<Severity> for ErrorSeverity {
+    fn from(value: Severity) -> Self {
+        match value {
+            Severity::Error => Self::Error,
+            Severity::Warning => Self::Warning,
+            Severity::Advice => Self::Advice,
+        }
+    }
+}
+
+#[ast]
+#[generate_derive(ESTree)]
+#[estree(no_type, no_ts_def)]
+pub struct ErrorLabel<'a> {
+    pub message: Option<Atom<'a>>,
+    pub span: Span,
+}
+
+impl<'a> FromIn<'a, &LabeledSpan> for ErrorLabel<'a> {
+    #[expect(clippy::cast_possible_truncation)]
+    fn from_in(label: &LabeledSpan, allocator: &'a Allocator) -> Self {
+        Self {
+            message: label.label().map(|message| Atom::from_in(message, allocator)),
+            span: Span::new(label.offset() as u32, (label.offset() + label.len()) as u32),
+        }
+    }
+}
+
+// Module record.
+//
+// These types and the `From` impl mirror the implementation in `types.rs` and `convert.rs`.
+// However, a lot of the data can be left as is, because it's already in the arena,
+// and that's what raw transfer needs - unlike the other implementation which requires owned types.
+// In particular, there's no need to copy or convert the many `Atom`s in `ModuleRecord`.
+
+#[ast]
+#[generate_derive(ESTree)]
+#[estree(no_type, no_ts_def)]
+pub struct EcmaScriptModule<'a> {
+    /// Has ESM syntax.
+    ///
+    /// i.e. `import` and `export` statements, and `import.meta`.
+    ///
+    /// Dynamic imports `import('foo')` are ignored since they can be used in non-ESM files.
+    pub has_module_syntax: bool,
+    /// Import statements.
+    pub static_imports: Vec<'a, StaticImport<'a>>,
+    /// Export statements.
+    pub static_exports: Vec<'a, StaticExport<'a>>,
+    /// Dynamic import expressions.
+    pub dynamic_imports: Vec<'a, DynamicImport>,
+    /// Span positions` of `import.meta`
+    pub import_metas: Vec<'a, Span>,
+}
+
+#[ast]
+#[generate_derive(ESTree)]
+#[estree(no_type, no_ts_def)]
+pub struct StaticImport<'a> {
+    /// Span of import statement.
+    pub span: Span,
+    /// Import source.
+    ///
+    /// ```js
+    /// import { foo } from "mod";
+    /// //                   ^^^
+    /// ```
+    pub module_request: NameSpan<'a>,
+    /// Import specifiers.
+    ///
+    /// Empty for `import "mod"`.
+    pub entries: Vec<'a, ImportEntry<'a>>,
+}
+
+#[ast]
+#[generate_derive(ESTree)]
+#[estree(no_type, no_ts_def)]
+pub struct StaticExport<'a> {
+    pub span: Span,
+    pub entries: Vec<'a, ExportEntry<'a>>,
+}
+
+impl<'a> FromIn<'a, ModuleRecord<'a>> for EcmaScriptModule<'a> {
+    fn from_in(record: ModuleRecord<'a>, allocator: &'a Allocator) -> Self {
+        let static_imports =
+            record.requested_modules.iter().flat_map(|(name, requested_modules)| {
+                requested_modules.iter().filter(|m| m.is_import).map(|m| {
+                    let entries = record
+                        .import_entries
+                        .iter()
+                        .filter(|e| e.statement_span == m.statement_span)
+                        .cloned();
+                    let entries = Vec::from_iter_in(entries, allocator);
+
+                    StaticImport {
+                        span: m.statement_span,
+                        module_request: NameSpan { name: *name, span: m.span },
+                        entries,
+                    }
+                })
+            });
+        let mut static_imports = Vec::from_iter_in(static_imports, allocator);
+        static_imports.sort_unstable_by_key(|e| e.span.start);
+
+        let static_exports = record
+            .local_export_entries
+            .iter()
+            .chain(&record.indirect_export_entries)
+            .chain(&record.star_export_entries)
+            .fold(FxHashMap::<Span, Vec<'a, ExportEntry>>::default(), |mut acc, e| {
+                acc.entry(e.statement_span)
+                    .or_insert_with(|| Vec::new_in(allocator))
+                    .push(e.clone());
+                acc
+            })
+            .into_iter()
+            .map(|(span, entries)| StaticExport { span, entries });
+        let mut static_exports = Vec::from_iter_in(static_exports, allocator);
+        static_exports.sort_unstable_by_key(|e| e.span.start);
+
+        Self {
+            has_module_syntax: record.has_module_syntax,
+            static_imports,
+            static_exports,
+            dynamic_imports: record.dynamic_imports,
+            import_metas: record.import_metas,
+        }
+    }
+}

napi/parser/test/parse-raw.test.ts

@@ -0,0 +1,102 @@
+import { readdir, readFile, writeFile } from 'node:fs/promises';
+import { basename, join as pathJoin } from 'node:path';
+import { describe, expect, it } from 'vitest';
+
+import { parseSync } from '../index.js';
+
+const TARGET_DIR_PATH = pathJoin(import.meta.dirname, '../../../target');
+const TEST262_DIR_PATH = pathJoin(import.meta.dirname, '../../../tasks/coverage/test262/test');
+const ACORN_TEST262_DIR_PATH = pathJoin(import.meta.dirname, '../../../tasks/coverage/acorn-test262/test');
+
+// Load/download fixtures.
+// Save in `target` directory, same as where benchmarks store them.
+const benchFixtureUrls = [
+  // TypeScript syntax (2.81MB)
+  'https://raw.githubusercontent.com/microsoft/TypeScript/v5.3.3/src/compiler/checker.ts',
+  // Real world app tsx (1.0M)
+  'https://raw.githubusercontent.com/oxc-project/benchmark-files/main/cal.com.tsx',
+  // Real world content-heavy app jsx (3K)
+  'https://raw.githubusercontent.com/oxc-project/benchmark-files/main/RadixUIAdoptionSection.jsx',
+  // Heavy with classes (554K)
+  'https://cdn.jsdelivr.net/npm/pdfjs-dist@4.0.269/build/pdf.mjs',
+  // ES5 (3.9M)
+  'https://cdn.jsdelivr.net/npm/antd@5.12.5/dist/antd.js',
+];
+
+const benchFixtures = await Promise.all(benchFixtureUrls.map(async (url) => {
+  const filename = url.split('/').at(-1),
+    path = pathJoin(TARGET_DIR_PATH, filename);
+  let sourceText;
+  try {
+    sourceText = await readFile(path, 'utf8');
+  } catch {
+    const res = await fetch(url);
+    sourceText = await res.text();
+    await writeFile(path, sourceText);
+  }
+
+  return [filename, sourceText];
+}));
+
+// Only test Test262 fixtures which Acorn is able to parse
+const test262FixturePaths = (await readdir(ACORN_TEST262_DIR_PATH, { recursive: true }))
+  .filter(path => path.endsWith('.json'))
+  .map(path => path.slice(0, -2));
+
+// Test raw transfer output matches standard (via JSON) output for some large files
+describe('fixtures', () => {
+  it.each(benchFixtures)('%s', testRaw, { timeout: 10000 });
+});
+
+// Test raw transfer output matches standard (via JSON) output for Test262 test cases
+describe('test262', () => {
+  it.each(test262FixturePaths)('%s', async (path) => {
+    const filename = basename(path);
+    const sourceText = await readFile(pathJoin(TEST262_DIR_PATH, path), 'utf8');
+    testRaw(filename, sourceText);
+  });
+});
+
+function testRaw(filename, sourceText) {
+  const retStandard = parseSync(filename, sourceText);
+  const { program: programStandard, comments: commentsStandard, module: moduleStandard, errors: errorsStandard } =
+    retStandard;
+
+  // @ts-ignore
+  const retRaw = parseSync(filename, sourceText, { experimentalRawTransfer: true });
+  const { program: programRaw, comments: commentsRaw } = retRaw;
+  // Remove `null` values, to match what NAPI-RS does
+  const moduleRaw = clean(retRaw.module);
+  const errorsRaw = clean(retRaw.errors);
+
+  // Compare as objects
+  expect(programRaw).toEqual(programStandard);
+  expect(commentsRaw).toEqual(commentsStandard);
+  expect(moduleRaw).toEqual(moduleStandard);
+  expect(errorsRaw).toEqual(errorsStandard);
+
+  // Compare as JSON (to ensure same field order)
+  const jsonStandard = stringify({
+    program: programStandard,
+    comments: commentsStandard,
+    module: moduleStandard,
+    errors: errorsStandard,
+  });
+  const jsonRaw = stringify({ program: programRaw, comments: commentsRaw, module: moduleRaw, errors: errorsRaw });
+  expect(jsonRaw).toEqual(jsonStandard);
+}
+
+// Stringify to JSON, removing values which are invalid in JSON
+function stringify(obj) {
+  return JSON.stringify(obj, (_key, value) => {
+    if (typeof value === 'bigint') return `__BIGINT__: ${value}`;
+    if (typeof value === 'object' && value instanceof RegExp) return `__REGEXP__: ${value}`;
+    if (value === Infinity) return `__INFINITY__`;
+    return value;
+  });
+}
+
+// Remove `null` values, to match what NAPI-RS does
+function clean(obj) {
+  return JSON.parse(JSON.stringify(obj, (_key, value) => value === null ? undefined : value));
+}

tasks/ast_tools/src/derives/estree.rs

@@ -187,6 +187,7 @@ fn parse_estree_attr(location: AttrLocation, part: AttrPart) -> Result<()> {
         // `#[estree]` attr on meta type
         AttrLocation::Meta(meta) => match part {
             AttrPart::String("ts_type", ts_type) => meta.estree.ts_type = Some(ts_type),
+            AttrPart::String("raw_deser", raw_deser) => meta.estree.raw_deser = Some(raw_deser),
             _ => return Err(()),
         },
         _ => unreachable!(),
@@ -466,7 +467,7 @@ fn generate_body_for_via_override(
 /// Returns `true` if either the field has an `#[estree(skip)]` attr on it,
 /// or the type that the field contains has an `#[estree(skip)]` attr.
 ///
-/// This function also used by Typescript generator.
+/// This function also used by Typescript and raw transfer generators.
 pub fn should_skip_field(field: &FieldDef, schema: &Schema) -> bool {
     if field.estree.skip {
         true
@@ -527,7 +528,7 @@ pub fn can_flatten_field_inline(field: &FieldDef, krate: &str, schema: &Schema)
 /// * `#[estree(rename)]` attr on variant.
 /// * `#[estree(no_rename_variants)]` attr on enum.
 ///
-/// This function also used by Typescript generator.
+/// This function also used by Typescript and raw transfer generators.
 pub fn get_fieldless_variant_value<'s>(
     enum_def: &'s EnumDef,
     variant: &'s VariantDef,
@@ -543,7 +544,7 @@ pub fn get_fieldless_variant_value<'s>(
 
 /// Get ESTree name for struct field.
 ///
-/// This function also used by Typescript generator.
+/// This function also used by Typescript and raw transfer generators.
 pub fn get_struct_field_name(field: &FieldDef) -> Cow<'_, str> {
     if let Some(field_name) = field.estree.rename.as_deref() {
         Cow::Borrowed(field_name)

tasks/ast_tools/src/generators/assert_layouts.rs

@@ -513,7 +513,10 @@ fn template(krate: &str, assertions_64: &TokenStream, assertions_32: &TokenStrea
             use nonmax::NonMaxU32;
 
             ///@@line_break
-            use crate::{number::*, operator::*, reference::*, scope::*, symbol::*};
+            use crate::{module_record::*, number::*, operator::*, reference::*, scope::*, symbol::*};
+        },
+        "napi/parser" => quote! {
+            use crate::raw_transfer_types::*;
         },
         _ => quote! {
             use crate::*;

tasks/ast_tools/src/generators/mod.rs

@@ -8,6 +8,7 @@ mod assert_layouts;
 mod ast_builder;
 mod ast_kind;
 mod get_id;
+mod raw_transfer;
 mod typescript;
 mod utf8_to_utf16;
 mod visit;
@@ -16,6 +17,7 @@ pub use assert_layouts::AssertLayouts;
 pub use ast_builder::AstBuilderGenerator;
 pub use ast_kind::AstKindGenerator;
 pub use get_id::GetIdGenerator;
+pub use raw_transfer::RawTransferGenerator;
 pub use typescript::TypescriptGenerator;
 pub use utf8_to_utf16::Utf8ToUtf16ConverterGenerator;
 pub use visit::VisitGenerator;

tasks/ast_tools/src/generators/raw_transfer.rs

@@ -0,0 +1,837 @@
+//! Generator for raw transfer deserializer.
+
+use std::{borrow::Cow, fmt::Debug, str};
+
+use cow_utils::CowUtils;
+use lazy_static::lazy_static;
+use regex::{Captures, Regex, Replacer};
+use rustc_hash::FxHashSet;
+
+use crate::{
+    Generator, RAW_TRANSFER_DESERIALIZER_PATH,
+    codegen::{Codegen, DeriveId},
+    derives::estree::{
+        get_fieldless_variant_value, get_struct_field_name, should_flatten_field, should_skip_field,
+    },
+    output::Output,
+    schema::{
+        BoxDef, CellDef, Def, EnumDef, FieldDef, OptionDef, PrimitiveDef, Schema, StructDef,
+        TypeDef, VecDef,
+        extensions::layout::{GetLayout, GetOffset},
+    },
+    utils::{FxIndexMap, format_cow, upper_case_first, write_it},
+};
+
+use super::define_generator;
+
+// Offsets of `Vec`'s fields.
+// These are based on observation, and are not stable.
+// They will be fully reliable once we implement our own `Vec` type and make it `#[repr(C)]`.
+const VEC_PTR_FIELD_OFFSET: usize = 0;
+const VEC_LEN_FIELD_OFFSET: usize = 24;
+
+/// Generator for raw transfer deserializer.
+pub struct RawTransferGenerator;
+
+define_generator!(RawTransferGenerator);
+
+impl Generator for RawTransferGenerator {
+    fn generate(&self, schema: &Schema, codegen: &Codegen) -> Output {
+        let code = generate_deserializers(schema, codegen);
+        Output::Javascript { path: RAW_TRANSFER_DESERIALIZER_PATH.to_string(), code }
+    }
+}
+
+/// Prelude to generated deserializer.
+/// Defines the main `deserialize` function.
+static PRELUDE: &str = "
+    'use strict';
+
+    module.exports = deserialize;
+
+    let uint8, uint32, float64, sourceText, sourceIsAscii, sourceLen;
+
+    const textDecoder = new TextDecoder('utf-8', { ignoreBOM: true }),
+        decodeStr = textDecoder.decode.bind(textDecoder),
+        { fromCodePoint } = String;
+
+    function deserialize(buffer, sourceTextInput, sourceLenInput) {
+        uint8 = buffer;
+        uint32 = new Uint32Array(buffer.buffer, buffer.byteOffset);
+        float64 = new Float64Array(buffer.buffer, buffer.byteOffset);
+
+        sourceText = sourceTextInput;
+        sourceLen = sourceLenInput;
+        sourceIsAscii = sourceText.length === sourceLen;
+
+        // (2 * 1024 * 1024 * 1024 - 16) >> 2
+        const metadataPos32 = 536870908;
+
+        const data = deserializeRawTransferData(uint32[metadataPos32]);
+
+        uint8 = uint32 = float64 = sourceText = undefined;
+
+        return data;
+    }
+";
+
+/// Generate deserializer functions for all types.
+fn generate_deserializers(schema: &Schema, codegen: &Codegen) -> String {
+    let estree_derive_id = codegen.get_derive_id_by_name("ESTree");
+
+    let mut code = PRELUDE.to_string();
+
+    for type_def in &schema.types {
+        match type_def {
+            TypeDef::Struct(struct_def) => {
+                generate_struct(struct_def, &mut code, estree_derive_id, schema);
+            }
+            TypeDef::Enum(enum_def) => {
+                generate_enum(enum_def, &mut code, estree_derive_id, schema);
+            }
+            TypeDef::Primitive(primitive_def) => {
+                generate_primitive(primitive_def, &mut code, schema);
+            }
+            TypeDef::Option(option_def) => {
+                generate_option(option_def, &mut code, schema);
+            }
+            TypeDef::Box(box_def) => {
+                generate_box(box_def, &mut code, schema);
+            }
+            TypeDef::Vec(vec_def) => {
+                generate_vec(vec_def, &mut code, schema);
+            }
+            TypeDef::Cell(_cell_def) => {
+                // No deserializers for `Cell`s - use inner type's deserializer
+            }
+        }
+    }
+
+    code
+}
+
+/// Generate deserialize function for a struct.
+fn generate_struct(
+    struct_def: &StructDef,
+    code: &mut String,
+    estree_derive_id: DeriveId,
+    schema: &Schema,
+) {
+    if !struct_def.generates_derive(estree_derive_id) || struct_def.estree.skip {
+        return;
+    }
+
+    let fn_name = struct_def.deser_name(schema);
+    let mut generator = StructDeserializerGenerator::new(schema);
+
+    let body = if let Some(converter_name) = &struct_def.estree.via {
+        generator.apply_converter(converter_name, struct_def, 0).map(|value| {
+            if generator.preamble.is_empty() {
+                format!("return {value};")
+            } else {
+                let preamble = generator.preamble.join("");
+                format!(
+                    "
+                        {preamble}
+                        return {value};
+                    "
+                )
+            }
+        })
+    } else {
+        None
+    };
+
+    let body = if let Some(body) = body {
+        body
+    } else {
+        let mut preamble_str = String::new();
+        let mut fields_str = String::new();
+
+        generator.generate_struct_fields(struct_def, 0);
+
+        for (field_name, value) in generator.fields {
+            if value.starts_with("...") {
+                write_it!(&mut fields_str, "{value},");
+            } else if generator.dependent_field_names.contains(&field_name) {
+                if preamble_str.is_empty() {
+                    preamble_str.push_str("const ");
+                } else {
+                    preamble_str.push_str(",\n");
+                }
+                write_it!(&mut preamble_str, "{field_name} = {value}");
+                write_it!(&mut fields_str, "{field_name},");
+            } else if value == field_name {
+                write_it!(&mut fields_str, "{field_name},");
+            } else {
+                write_it!(&mut fields_str, "{field_name}: {value},");
+            }
+        }
+
+        if !preamble_str.is_empty() {
+            preamble_str.push(';');
+        }
+
+        for preamble_part in generator.preamble {
+            preamble_str.push_str(preamble_part.trim());
+        }
+
+        format!(
+            "
+            {preamble_str}
+            return {{
+                {fields_str}
+            }};
+        "
+        )
+    };
+
+    #[rustfmt::skip]
+    write_it!(code, "
+        function {fn_name}(pos) {{
+            {body}
+        }}
+    ");
+}
+
+struct StructDeserializerGenerator<'s> {
+    /// Dependencies
+    dependent_field_names: FxHashSet<String>,
+    /// Preamble
+    preamble: Vec<String>,
+    /// Fields, keyed by fields name (field name in ESTree AST)
+    fields: FxIndexMap<String, String>,
+    /// Schema
+    schema: &'s Schema,
+}
+
+impl<'s> StructDeserializerGenerator<'s> {
+    fn new(schema: &'s Schema) -> Self {
+        Self {
+            dependent_field_names: FxHashSet::default(),
+            preamble: vec![],
+            fields: FxIndexMap::default(),
+            schema,
+        }
+    }
+
+    fn generate_struct_fields(&mut self, struct_def: &StructDef, struct_offset: u32) {
+        if let Some(field_indices) = &struct_def.estree.field_indices {
+            // Specified field order - serialize in this order
+            for &field_index in field_indices {
+                let field_index = field_index as usize;
+                if let Some(field) = struct_def.fields.get(field_index) {
+                    self.generate_struct_field_owned(field, struct_def, struct_offset);
+                } else {
+                    let (field_name, converter_name) =
+                        &struct_def.estree.add_fields[field_index - struct_def.fields.len()];
+                    self.generate_struct_field_added(
+                        struct_def,
+                        field_name,
+                        converter_name,
+                        struct_offset,
+                    );
+                }
+            }
+        } else {
+            // No specified field order - serialize in original order
+            for field in &struct_def.fields {
+                self.generate_struct_field_owned(field, struct_def, struct_offset);
+            }
+
+            for (field_name, converter_name) in &struct_def.estree.add_fields {
+                self.generate_struct_field_added(
+                    struct_def,
+                    field_name,
+                    converter_name,
+                    struct_offset,
+                );
+            }
+        }
+
+        // Add `type` field if there isn't one already, and struct isn't marked `#[estree(no_type)]`
+        if !struct_def.estree.no_type && !self.fields.contains_key("type") {
+            let struct_name =
+                struct_def.estree.rename.as_deref().unwrap_or_else(|| struct_def.name());
+            self.fields.insert_before(0, "type".to_string(), format!("'{struct_name}'"));
+        }
+    }
+
+    fn generate_struct_field_owned(
+        &mut self,
+        field: &FieldDef,
+        struct_def: &StructDef,
+        struct_offset: u32,
+    ) {
+        if should_skip_field(field, self.schema) {
+            return;
+        }
+
+        let field_name = get_struct_field_name(field).to_string();
+        let field_type = field.type_def(self.schema);
+        let field_offset = struct_offset + field.offset_64();
+
+        if should_flatten_field(field, self.schema) {
+            match field_type {
+                TypeDef::Struct(field_struct_def) => {
+                    self.generate_struct_fields(field_struct_def, field_offset);
+                }
+                TypeDef::Enum(field_enum_def) => {
+                    // TODO: Do this better
+                    let value_fn = field_enum_def.deser_name(self.schema);
+                    let pos = pos_offset(field_offset);
+                    self.fields.insert(field_name, format!("...{value_fn}({pos})"));
+                }
+                _ => panic!(
+                    "Cannot flatten a field which is not a struct or enum: `{}::{}`",
+                    struct_def.name(),
+                    field.name(),
+                ),
+            }
+            return;
+        }
+
+        let value_fn = field_type.deser_name(self.schema);
+        let pos = pos_offset(field_offset);
+        let mut value = format!("{value_fn}({pos})");
+
+        if let Some(appended_field_index) = field.estree.append_field_index {
+            self.preamble.push(format!("const {field_name} = {value};"));
+
+            let appended_field = &struct_def.fields[appended_field_index];
+            let appended_field_type = appended_field.type_def(self.schema);
+            match appended_field_type {
+                TypeDef::Vec(vec_def) => {
+                    let appended_field_fn = vec_def.deser_name(self.schema);
+                    let appended_pos = pos_offset(struct_offset + appended_field.offset_64());
+                    self.preamble.push(format!(
+                        "{field_name}.push(...{appended_field_fn}({appended_pos}));"
+                    ));
+                }
+                TypeDef::Option(option_def) => {
+                    let appended_field_name = get_struct_field_name(appended_field).to_string();
+                    let appended_field_fn = option_def.deser_name(self.schema);
+                    let appended_pos = pos_offset(struct_offset + appended_field.offset_64());
+                    self.preamble.push(format!("
+                        const {appended_field_name} = {appended_field_fn}({appended_pos});
+                        if ({appended_field_name} !== null) {field_name}.push({appended_field_name});
+                    "));
+                }
+                _ => panic!("Cannot append: `{}::{}`", struct_def.name(), field.name()),
+            }
+
+            value.clone_from(&field_name);
+        } else if let Some(converter_name) = &field.estree.via {
+            value = self.apply_converter(converter_name, struct_def, struct_offset).unwrap();
+        }
+
+        self.fields.insert(field_name, value);
+    }
+
+    fn generate_struct_field_added(
+        &mut self,
+        struct_def: &StructDef,
+        field_name: &str,
+        converter_name: &str,
+        struct_offset: u32,
+    ) {
+        let value = self.apply_converter(converter_name, struct_def, struct_offset).unwrap();
+        self.fields.insert(field_name.to_string(), value);
+    }
+
+    fn apply_converter(
+        &mut self,
+        converter_name: &str,
+        struct_def: &StructDef,
+        struct_offset: u32,
+    ) -> Option<String> {
+        let converter = self.schema.meta_by_name(converter_name);
+        let raw_deser = converter.estree.raw_deser.as_deref()?;
+
+        let value = THIS_REGEX.replace_all(raw_deser, ThisReplacer::new(self));
+        let value = DESER_REGEX.replace_all(&value, DeserReplacer::new(self.schema));
+        let value = POS_OFFSET_REGEX
+            .replace_all(&value, PosOffsetReplacer::new(self, struct_def, struct_offset));
+        let value = value.cow_replace("SOURCE_TEXT", "sourceText");
+        let value = if let Some((preamble, value)) = value.trim().rsplit_once('\n') {
+            self.preamble.push(preamble.to_string());
+            value.trim().to_string()
+        } else {
+            value.trim().to_string()
+        };
+        Some(value)
+    }
+}
+
+/// Generate deserialize function for an enum.
+fn generate_enum(
+    enum_def: &EnumDef,
+    code: &mut String,
+    estree_derive_id: DeriveId,
+    schema: &Schema,
+) {
+    if !enum_def.generates_derive(estree_derive_id) || enum_def.estree.skip {
+        return;
+    }
+
+    let type_name = enum_def.name();
+    let fn_name = enum_def.deser_name(schema);
+    let payload_offset = enum_def.layout_64().align;
+
+    let body = if let Some(converter_name) = &enum_def.estree.via {
+        apply_converter_for_enum(converter_name, 0, schema)
+    } else {
+        None
+    };
+
+    let body = body.unwrap_or_else(|| {
+        let mut variants = enum_def.all_variants(schema).collect::<Vec<_>>();
+        variants.sort_by_key(|variant| variant.discriminant);
+
+        let switch_cases = variants.into_iter().fold(String::new(), |mut s, variant| {
+            let discriminant = variant.discriminant;
+
+            let ret = if let Some(converter_name) = &variant.estree.via {
+                apply_converter_for_enum(converter_name, payload_offset, schema).unwrap()
+            } else if let Some(variant_type) = variant.field_type(schema) {
+                let variant_fn_name = variant_type.deser_name(schema);
+                let payload_pos = pos_offset(payload_offset);
+                format!("return {variant_fn_name}({payload_pos});")
+            } else {
+                format!("return '{}';", get_fieldless_variant_value(enum_def, variant))
+            };
+
+            write_it!(s, "case {discriminant}: {ret}");
+            s
+        });
+
+        format!(
+            "
+            switch(uint8[pos]) {{
+                {switch_cases}
+                default: throw new Error(`Unexpected discriminant ${{uint8[pos]}} for {type_name}`);
+            }}
+            "
+        )
+    });
+
+    #[rustfmt::skip]
+    write_it!(code, "
+        function {fn_name}(pos) {{
+            {body}
+        }}
+    ");
+}
+
+/// Generate deserialize function for a primitive.
+fn generate_primitive(primitive_def: &PrimitiveDef, code: &mut String, schema: &Schema) {
+    #[expect(clippy::match_same_arms)]
+    let ret = match primitive_def.name() {
+        // Reuse deserializer for `&str`
+        "Atom" => return,
+        // Dummy type
+        "PointerAlign" => return,
+        "bool" => "return uint8[pos] === 1;",
+        "u8" => "return uint8[pos];",
+        // "u16" => "return uint16[pos >> 1];",
+        "u32" => "return uint32[pos >> 2];",
+        #[rustfmt::skip]
+        "u64" => "
+            const pos32 = pos >> 2;
+            return uint32[pos32] + uint32[pos32 + 1] * 4294967296;
+        ",
+        "f64" => "return float64[pos >> 3];",
+        "&str" => STR_DESERIALIZER_BODY,
+        // Reuse deserializers for zeroed types
+        type_name if type_name.starts_with("NonZero") => return,
+        type_name => panic!("Cannot generate deserializer for primitive `{type_name}`"),
+    };
+
+    let fn_name = primitive_def.deser_name(schema);
+
+    #[rustfmt::skip]
+    write_it!(code, "
+        function {fn_name}(pos) {{
+            {ret}
+        }}
+    ");
+}
+
+static STR_DESERIALIZER_BODY: &str = "
+    const pos32 = pos >> 2,
+        len = uint32[pos32 + 2];
+    if (len === 0) return '';
+
+    pos = uint32[pos32];
+    if (sourceIsAscii && pos < sourceLen) return sourceText.substr(pos, len);
+
+    // Longer strings use `TextDecoder`
+    // TODO: Find best switch-over point
+    const end = pos + len;
+    if (len > 50) return decodeStr(uint8.subarray(pos, end));
+
+    // Shorter strings decode by hand to avoid native call
+    let out = '',
+        c;
+    do {
+        c = uint8[pos++];
+        if (c < 0x80) {
+            out += fromCodePoint(c);
+        } else {
+            out += decodeStr(uint8.subarray(pos - 1, end));
+            break;
+        }
+    } while (pos < end);
+
+    return out;
+";
+
+/// Generate deserialize function for an `Option`.
+fn generate_option(option_def: &OptionDef, code: &mut String, schema: &Schema) {
+    let fn_name = option_def.deser_name(schema);
+    let inner_type = option_def.inner_type(schema);
+    let inner_fn_name = inner_type.deser_name(schema);
+    let inner_layout = inner_type.layout_64();
+
+    let (none_condition, payload_offset) = if option_def.layout_64().size == inner_layout.size {
+        let niche = inner_layout.niche.clone().unwrap();
+        let none_condition = match niche.size {
+            1 => format!("uint8[{}] === {}", pos_offset(niche.offset), niche.value()),
+            // 2 => format!("uint16[{}] === {}", pos_offset_shift(niche.offset, 1), niche.value()),
+            4 => format!("uint32[{}] === {}", pos_offset_shift(niche.offset, 2), niche.value()),
+            8 => {
+                // TODO: Use `float64[pos >> 3] === 0` instead of
+                // `uint32[pos >> 2] === 0 && uint32[(pos + 4) >> 2] === 0`?
+                let value = niche.value();
+                format!(
+                    "uint32[{}] === {} && uint32[{}] === {}",
+                    pos_offset_shift(niche.offset, 2),
+                    value & u128::from(u32::MAX),
+                    pos_offset_shift(niche.offset + 4, 2),
+                    value >> 32,
+                )
+            }
+            size => panic!("Invalid niche size: {size}"),
+        };
+        (none_condition, Cow::Borrowed("pos"))
+    } else {
+        ("uint8[pos] === 0".to_string(), pos_offset(inner_layout.align))
+    };
+
+    #[rustfmt::skip]
+    write_it!(code, "
+        function {fn_name}(pos) {{
+            if ({none_condition}) return null;
+            return {inner_fn_name}({payload_offset});
+        }}
+    ");
+}
+
+/// Generate deserialize function for a `Box`.
+fn generate_box(box_def: &BoxDef, code: &mut String, schema: &Schema) {
+    let fn_name = box_def.deser_name(schema);
+    let inner_fn_name = box_def.inner_type(schema).deser_name(schema);
+
+    #[rustfmt::skip]
+    write_it!(code, "
+        function {fn_name}(pos) {{
+            return {inner_fn_name}(uint32[pos >> 2]);
+        }}
+    ");
+}
+
+/// Generate deserialize function for a `Vec`.
+fn generate_vec(vec_def: &VecDef, code: &mut String, schema: &Schema) {
+    let fn_name = vec_def.deser_name(schema);
+    let inner_type = vec_def.inner_type(schema);
+    let inner_fn_name = inner_type.deser_name(schema);
+    let inner_type_size = inner_type.layout_64().size;
+
+    let ptr_pos32 = pos32_offset(VEC_PTR_FIELD_OFFSET);
+    let len_pos32 = pos32_offset(VEC_LEN_FIELD_OFFSET);
+
+    #[rustfmt::skip]
+    write_it!(code, "
+        function {fn_name}(pos) {{
+            const arr = [],
+                pos32 = pos >> 2,
+                len = uint32[{len_pos32}];
+            pos = uint32[{ptr_pos32}];
+            for (let i = 0; i < len; i++) {{
+                arr.push({inner_fn_name}(pos));
+                pos += {inner_type_size};
+            }}
+            return arr;
+        }}
+    ");
+}
+
+/// Generate pos offset string.
+///
+/// * If `offset == 0` -> `pos`.
+/// * Otherwise -> `pos + <offset>` (e.g. `pos + 8`).
+fn pos_offset<O>(offset: O) -> Cow<'static, str>
+where
+    O: TryInto<u64>,
+    <O as TryInto<u64>>::Error: Debug,
+{
+    let offset = offset.try_into().unwrap();
+    if offset == 0 { Cow::Borrowed("pos") } else { format_cow!("pos + {offset}") }
+}
+
+/// Generate pos offset and shift string.
+///
+/// * If `offset == 0` and `shift == 0` -> `pos`.
+/// * If `offset == 0` -> `pos >> <shift>` (e.g. `pos >> 2`).
+/// * If `shift == 0` -> `pos + <offset>` (e.g. `pos + 8`).
+/// * Otherwise -> `(pos + <offset>) >> <shift>` (e.g. `(pos + 8) >> 2`).
+fn pos_offset_shift<O, S>(offset: O, shift: S) -> Cow<'static, str>
+where
+    O: TryInto<u64>,
+    <O as TryInto<u64>>::Error: Debug,
+    S: TryInto<u64>,
+    <S as TryInto<u64>>::Error: Debug,
+{
+    let offset = offset.try_into().unwrap();
+    let shift = shift.try_into().unwrap();
+    match (offset, shift) {
+        (0, 0) => Cow::Borrowed("pos"),
+        (0, _) => format_cow!("pos >> {shift}"),
+        (_, 0) => format_cow!("pos + {offset}"),
+        (_, _) => format_cow!("(pos + {offset}) >> {shift}"),
+    }
+}
+
+/// Generate pos32 offset string.
+///
+/// * If `offset == 0` -> `pos32`.
+/// * Otherwise -> `pos32 + <offset>` (e.g. `pos32 + 4`).
+fn pos32_offset<O>(offset: O) -> Cow<'static, str>
+where
+    O: TryInto<u64>,
+    <O as TryInto<u64>>::Error: Debug,
+{
+    let offset = offset.try_into().unwrap();
+    let offset32 = offset >> 2;
+    if offset32 == 0 { Cow::Borrowed("pos32") } else { format_cow!("pos32 + {offset32}") }
+}
+
+// `raw_deser` replacements
+
+/// Get `raw_deser` for converter and replace `POS`, `DESER` and `SOURCE_TEXT` within it.
+///
+/// Returns `None` if converter is not annotated `#[estree(raw_deser = "...")]`.
+fn apply_converter_for_enum(converter_name: &str, offset: u32, schema: &Schema) -> Option<String> {
+    let converter = schema.meta_by_name(converter_name);
+    let raw_deser = converter.estree.raw_deser.as_deref()?;
+
+    let value = POS_REGEX.replace_all(raw_deser, PosReplacer::new(offset));
+    let value = DESER_REGEX.replace_all(&value, DeserReplacer::new(schema));
+    let value = value.cow_replace("SOURCE_TEXT", "sourceText");
+    let value = if let Some((preamble, value)) = value.trim().rsplit_once('\n') {
+        format!("{preamble} return {value};")
+    } else {
+        format!("return {value};")
+    };
+
+    Some(value)
+}
+
+lazy_static! {
+    static ref THIS_REGEX: Regex = Regex::new(r"THIS\.([a-zA-Z_]+)").unwrap();
+}
+
+struct ThisReplacer<'d> {
+    dependent_field_names: &'d mut FxHashSet<String>,
+}
+
+impl<'d> ThisReplacer<'d> {
+    fn new(generator: &'d mut StructDeserializerGenerator<'_>) -> Self {
+        Self { dependent_field_names: &mut generator.dependent_field_names }
+    }
+}
+
+impl Replacer for ThisReplacer<'_> {
+    fn replace_append(&mut self, caps: &Captures, dst: &mut String) {
+        assert_eq!(caps.len(), 2);
+        let field_name = caps.get(1).unwrap().as_str();
+        dst.push_str(field_name);
+        self.dependent_field_names.insert(field_name.to_string());
+    }
+}
+
+lazy_static! {
+    static ref DESER_REGEX: Regex = Regex::new(r"DESER\[([A-Za-z0-9<>_]+)\]").unwrap();
+}
+
+struct DeserReplacer<'s> {
+    schema: &'s Schema,
+}
+
+impl<'s> DeserReplacer<'s> {
+    fn new(schema: &'s Schema) -> Self {
+        Self { schema }
+    }
+}
+
+impl Replacer for DeserReplacer<'_> {
+    fn replace_append(&mut self, caps: &Captures, dst: &mut String) {
+        assert_eq!(caps.len(), 2);
+        let type_name = caps.get(1).unwrap().as_str();
+        let type_def = self.schema.type_by_name(type_name);
+        let fn_name = type_def.deser_name(self.schema);
+        dst.push_str(&fn_name);
+    }
+}
+
+lazy_static! {
+    static ref POS_REGEX: Regex = {
+        #[expect(clippy::trivial_regex)]
+        Regex::new("POS").unwrap()
+    };
+}
+
+struct PosReplacer {
+    offset: u32,
+}
+
+impl PosReplacer {
+    fn new(offset: u32) -> Self {
+        Self { offset }
+    }
+}
+
+impl Replacer for PosReplacer {
+    fn replace_append(&mut self, _caps: &Captures, dst: &mut String) {
+        dst.push_str(&pos_offset(self.offset));
+    }
+}
+
+lazy_static! {
+    static ref POS_OFFSET_REGEX: Regex = Regex::new(
+        r"POS_OFFSET(?:<([A-Za-z]+)>)?\.([a-zA-Z_]+(?:\.[a-zA-Z_]+)*)(?:\s*\+\s*(\d+))?"
+    )
+    .unwrap();
+}
+
+struct PosOffsetReplacer<'s, 'd> {
+    schema: &'s Schema,
+    struct_def: &'d StructDef,
+    struct_offset: u32,
+}
+
+impl<'s, 'd> PosOffsetReplacer<'s, 'd> {
+    fn new(
+        generator: &StructDeserializerGenerator<'s>,
+        struct_def: &'d StructDef,
+        struct_offset: u32,
+    ) -> Self {
+        Self { schema: generator.schema, struct_def, struct_offset }
+    }
+}
+
+impl Replacer for PosOffsetReplacer<'_, '_> {
+    fn replace_append(&mut self, caps: &Captures, dst: &mut String) {
+        assert_eq!(caps.len(), 4);
+
+        let struct_def = if let Some(struct_name) = caps.get(1) {
+            self.schema.type_by_name(struct_name.as_str()).as_struct().unwrap()
+        } else {
+            self.struct_def
+        };
+
+        let mut field_names = caps.get(2).unwrap().as_str().split('.');
+        let field_name = field_names.next().unwrap();
+        let field = struct_def.fields.iter().find(|field| field.name() == field_name).unwrap();
+        let mut offset = self.struct_offset + field.offset_64();
+        let mut type_def = field.type_def(self.schema);
+        for field_name in field_names {
+            let struct_def = type_def.as_struct().unwrap();
+            let field = struct_def.fields.iter().find(|field| field.name() == field_name).unwrap();
+            offset += field.offset_64();
+            type_def = field.type_def(self.schema);
+        }
+
+        if let Some(add) = caps.get(3) {
+            offset += str::parse::<u32>(add.as_str()).unwrap();
+        }
+
+        if offset == 0 {
+            write_it!(dst, "pos");
+        } else {
+            write_it!(dst, "pos + {offset}");
+        }
+    }
+}
+
+/// Trait to get deserializer function name for a type.
+trait DeserializeFunctionName {
+    fn deser_name(&self, schema: &Schema) -> String {
+        format!("deserialize{}", self.plain_name(schema))
+    }
+
+    fn plain_name<'s>(&'s self, schema: &'s Schema) -> Cow<'s, str>;
+}
+
+impl DeserializeFunctionName for TypeDef {
+    fn plain_name<'s>(&'s self, schema: &'s Schema) -> Cow<'s, str> {
+        match self {
+            TypeDef::Struct(def) => def.plain_name(schema),
+            TypeDef::Enum(def) => def.plain_name(schema),
+            TypeDef::Primitive(def) => def.plain_name(schema),
+            TypeDef::Option(def) => def.plain_name(schema),
+            TypeDef::Box(def) => def.plain_name(schema),
+            TypeDef::Vec(def) => def.plain_name(schema),
+            TypeDef::Cell(def) => def.plain_name(schema),
+        }
+    }
+}
+
+macro_rules! impl_deser_name_simple {
+    ($ty:ident) => {
+        impl DeserializeFunctionName for $ty {
+            fn plain_name<'s>(&'s self, _schema: &'s Schema) -> Cow<'s, str> {
+                Cow::Borrowed(self.name())
+            }
+        }
+    };
+}
+
+impl_deser_name_simple!(StructDef);
+impl_deser_name_simple!(EnumDef);
+
+macro_rules! impl_deser_name_concat {
+    ($ty:ident, $prefix:expr) => {
+        impl DeserializeFunctionName for $ty {
+            fn plain_name<'s>(&'s self, schema: &'s Schema) -> Cow<'s, str> {
+                format_cow!("{}{}", $prefix, self.inner_type(schema).plain_name(schema))
+            }
+        }
+    };
+}
+
+impl_deser_name_concat!(OptionDef, "Option");
+impl_deser_name_concat!(BoxDef, "Box");
+impl_deser_name_concat!(VecDef, "Vec");
+
+impl DeserializeFunctionName for PrimitiveDef {
+    fn plain_name<'s>(&'s self, _schema: &'s Schema) -> Cow<'s, str> {
+        let type_name = self.name();
+        if matches!(type_name, "&str" | "Atom") {
+            // Use 1 deserializer for both `&str` and `Atom`
+            Cow::Borrowed("Str")
+        } else if let Some(type_name) = type_name.strip_prefix("NonZero") {
+            // Use zeroed type's deserializer for `NonZero*` types
+            Cow::Borrowed(type_name)
+        } else {
+            upper_case_first(type_name)
+        }
+    }
+}
+
+impl DeserializeFunctionName for CellDef {
+    fn plain_name<'s>(&'s self, schema: &'s Schema) -> Cow<'s, str> {
+        // `Cell`s use same deserializer as inner type, as layout is identical
+        self.inner_type(schema).plain_name(schema)
+    }
+}

tasks/ast_tools/src/generators/utf8_to_utf16.rs

@@ -43,6 +43,9 @@ impl Generator for Utf8ToUtf16ConverterGenerator {
 fn generate(schema: &Schema, codegen: &Codegen) -> TokenStream {
     let estree_derive_id = codegen.get_derive_id_by_name("ESTree");
     let span_type_id = schema.type_names["Span"];
+
+    // Types with custom visitors (see comment above).
+    // Also skip `Comment` because we handle adjusting comment spans separately.
     let skip_type_ids = [
         "ObjectProperty",
         "BindingProperty",
@@ -50,6 +53,7 @@ fn generate(schema: &Schema, codegen: &Codegen) -> TokenStream {
         "ExportSpecifier",
         "WithClause",
         "TemplateLiteral",
+        "Comment",
     ]
     .map(|type_name| schema.type_names[type_name]);
 
@@ -64,8 +68,12 @@ fn generate(schema: &Schema, codegen: &Codegen) -> TokenStream {
             return None;
         }
 
-        // Skip `oxc_regular_expression` types. They don't appear in ESTree AST.
-        if struct_def.file(schema).krate() == "oxc_regular_expression" {
+        // Skip types in `oxc_regular_expression`, `oxc_syntax`, and `napi/parser` crates.
+        // They don't appear in ESTree AST.
+        if matches!(
+            struct_def.file(schema).krate(),
+            "oxc_regular_expression" | "oxc_syntax" | "napi/parser"
+        ) {
             return None;
         }
 

tasks/ast_tools/src/main.rs

@@ -214,14 +214,17 @@ static SOURCE_PATHS: &[&str] = &[
     "crates/oxc_ast/src/ast/comment.rs",
     "crates/oxc_ast/src/serialize.rs",
     "crates/oxc_syntax/src/lib.rs",
+    "crates/oxc_syntax/src/module_record.rs",
     "crates/oxc_syntax/src/number.rs",
     "crates/oxc_syntax/src/operator.rs",
     "crates/oxc_syntax/src/scope.rs",
+    "crates/oxc_syntax/src/serialize.rs",
     "crates/oxc_syntax/src/symbol.rs",
     "crates/oxc_syntax/src/reference.rs",
     "crates/oxc_span/src/span.rs",
     "crates/oxc_span/src/source_type/mod.rs",
     "crates/oxc_regular_expression/src/ast.rs",
+    "napi/parser/src/raw_transfer_types.rs",
 ];
 
 /// Path to `oxc_ast` crate
@@ -236,6 +239,9 @@ const AST_MACROS_CRATE_PATH: &str = "crates/oxc_ast_macros";
 /// Path to write TS type definitions to
 const TYPESCRIPT_DEFINITIONS_PATH: &str = "npm/oxc-types/types.d.ts";
 
+/// Path to write raw deserializer to
+const RAW_TRANSFER_DESERIALIZER_PATH: &str = "napi/parser/deserialize.js";
+
 /// Path to write CI filter list to
 const GITHUB_WATCH_LIST_PATH: &str = ".github/.generated_ast_watch_list.yml";
 
@@ -257,6 +263,7 @@ const GENERATORS: &[&(dyn Generator + Sync)] = &[
     &generators::GetIdGenerator,
     &generators::VisitGenerator,
     &generators::Utf8ToUtf16ConverterGenerator,
+    &generators::RawTransferGenerator,
     &generators::TypescriptGenerator,
 ];
 

tasks/ast_tools/src/schema/extensions/estree.rs

@@ -74,4 +74,6 @@ pub struct ESTreeEnumVariant {
 #[derive(Default, Debug)]
 pub struct ESTreeMeta {
     pub ts_type: Option<String>,
+    // JS code for raw transfer deserializer
+    pub raw_deser: Option<String>,
 }

tasks/ast_tools/src/schema/extensions/layout.rs

@@ -101,7 +101,6 @@ impl Niche {
     }
 
     /// Get value of the [`Niche`].
-    #[expect(unused)]
     pub fn value(&self) -> u128 {
         // Prefer to consume niches at start of range over end of range
         if self.count_start > 0 {

tasks/ast_tools/src/schema/mod.rs

@@ -64,7 +64,6 @@ impl Schema {
     ///
     /// # Panics
     /// Panics if no type with supplied name.
-    #[expect(dead_code)]
     pub fn type_by_name(&self, name: &str) -> &TypeDef {
         let type_id = self.type_names[name];
         &self.types[type_id]

tasks/ast_tools/src/utils.rs

@@ -1,3 +1,5 @@
+use std::borrow::Cow;
+
 use indexmap::{IndexMap, IndexSet};
 use phf::{Set as PhfSet, phf_set};
 use proc_macro2::{Span, TokenStream};
@@ -83,6 +85,17 @@ pub fn pluralize(name: &str) -> String {
     }
 }
 
+/// Upper case first character of a string.
+pub fn upper_case_first(s: &str) -> Cow<'_, str> {
+    let mut chars = s.chars();
+    let first_char = chars.next().unwrap();
+    if first_char.is_uppercase() {
+        Cow::Borrowed(s)
+    } else {
+        Cow::Owned(first_char.to_uppercase().chain(chars).collect::<String>())
+    }
+}
+
 /// Macro to `format!` arguments, and wrap the formatted string in a `Cow::Owned`.
 macro_rules! format_cow {
     ($($tokens:tt)+) => {