mirror of
https://github.com/oxc-project/oxc.git
synced 2026-11-04 10:29:58 +00:00
fix(formatter_markdown): preserve line breaks around Chinese/Japanese characters in all proseWrap (#27331)
Align with Prettier 3.10 (not released) and also dprint and biome.
This commit is contained in:
1 parent
e2c68b17fa
commit
7d0e54dc20
16 files changed
+514
-486
No files matched your search
@@ -245,7 +245,7 @@
|
||||
- [externals/prettier/markdown/thematicBreak/simple.md](diffs/markdown/externals__prettier__markdown__thematicBreak__simple.md.md)
|
||||
- crates/oxc_formatter_markdown/DIVERGENCES.md#leading-thematic-break
|
||||
|
||||
### Option 2: 270/282 (95.74%)
|
||||
### Option 2: 269/282 (95.39%)
|
||||
|
||||
```json
|
||||
{"printWidth":100,"proseWrap":"always"}
|
||||
@@ -273,6 +273,8 @@
|
||||
- crates/oxc_formatter/DIVERGENCES.md#suppressed-terminator-per-semi
|
||||
- [externals/prettier/markdown/paragraph/cjk.md](diffs/markdown/externals__prettier__markdown__paragraph__cjk.md.md)
|
||||
- crates/oxc_formatter_markdown/DIVERGENCES.md#container-directive
|
||||
- [externals/prettier/markdown/splitCjkText/symbolSpaceNewLine.md](diffs/markdown/externals__prettier__markdown__splitCjkText__symbolSpaceNewLine.md.md)
|
||||
- crates/oxc_formatter_markdown/DIVERGENCES.md#cj-line-break
|
||||
- [externals/prettier/markdown/thematicBreak/simple.md](diffs/markdown/externals__prettier__markdown__thematicBreak__simple.md.md)
|
||||
- crates/oxc_formatter_markdown/DIVERGENCES.md#leading-thematic-break
|
||||
|
||||
|
||||
+179
@@ -0,0 +1,179 @@
|
||||
# externals/prettier/markdown/splitCjkText/symbolSpaceNewLine.md
|
||||
|
||||
## Option 2
|
||||
|
||||
`````json
|
||||
{"printWidth":100,"proseWrap":"always"}
|
||||
`````
|
||||
|
||||
### Diff
|
||||
|
||||
`````diff
|
||||
===================================================================
|
||||
--- prettier
|
||||
+++ oxfmt
|
||||
@@ -1,14 +1,65 @@
|
||||
-日本語、にほんご。汉语, 中文. 日本語,にほんご.English
|
||||
-words!? 漢字!汉字?「セリフ」(括弧) 文字(括弧)文字【括弧】日本語English
|
||||
+日本語
|
||||
+、
|
||||
+にほんご
|
||||
+。
|
||||
+汉语,
|
||||
+中文.
|
||||
+日
|
||||
+本
|
||||
+語
|
||||
+,
|
||||
+に
|
||||
+ほ
|
||||
+ん
|
||||
+ご
|
||||
+.
|
||||
+English words!?
|
||||
+漢字
|
||||
+!
|
||||
+汉字
|
||||
+?
|
||||
+「セリフ」
|
||||
+(括弧)
|
||||
+文字
|
||||
+(括弧)
|
||||
+文字
|
||||
+【括弧】
|
||||
+日本語
|
||||
+English
|
||||
|
||||
-「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)」「禁則(きんそく)処理(しょり)」「禁則(きんそく)処理(しょり)!!!!」「禁則(きんそく)処理(しょり)!!!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」
|
||||
+「禁則(きんそく)処理(しょり)!」
|
||||
+「禁則(きんそく)処理(しょり)!」
|
||||
+「禁則(きんそく)処理(しょり)!」
|
||||
+「禁則(きんそく)処理(しょり)」
|
||||
+「禁則(きんそく)処理(しょり)!」
|
||||
+「禁則(きんそく)処理(しょり)!」
|
||||
+「禁則(きんそく)処理(しょり)」
|
||||
+「禁則(きんそく)処理(しょり)」
|
||||
+「禁則(きんそく)処理(しょり)!!!!」
|
||||
+「禁則(きんそく)処理(しょり)!!!」
|
||||
+「禁則(きんそく)処理(しょり)!」
|
||||
+「禁則(きんそく)処理(しょり)!」
|
||||
+「禁則(きんそく)処理(しょり)!」
|
||||
+「禁則(きんそく)処理(しょり)!」
|
||||
+「禁則(きんそく)処理(しょり)!」
|
||||
+「禁則(きんそく)処理(しょり)!」
|
||||
|
||||
-中点・中点
|
||||
+中点
|
||||
+・
|
||||
+中点
|
||||
|
||||
-禁則処理にわざと違反した文章のテストを今から行います。準備はいいでしょうか?レディ、ゴー!
|
||||
+禁則処理にわざと違反した文章のテストを今から行います。準備はいいでしょうか?レデ
|
||||
+ィ、ゴー!
|
||||
|
||||
[ウ ィキペディア]
|
||||
|
||||
[ウ ィキペディア]: https://ja.wikipedia.org/
|
||||
|
||||
-C言語・C++・Go・Rust
|
||||
+C言
|
||||
+語
|
||||
+・
|
||||
+C++
|
||||
+・
|
||||
+Go
|
||||
+・
|
||||
+Rust
|
||||
|
||||
`````
|
||||
|
||||
### Actual (oxfmt)
|
||||
|
||||
`````md
|
||||
日本語
|
||||
、
|
||||
にほんご
|
||||
。
|
||||
汉语,
|
||||
中文.
|
||||
日
|
||||
本
|
||||
語
|
||||
,
|
||||
に
|
||||
ほ
|
||||
ん
|
||||
ご
|
||||
.
|
||||
English words!?
|
||||
漢字
|
||||
!
|
||||
汉字
|
||||
?
|
||||
「セリフ」
|
||||
(括弧)
|
||||
文字
|
||||
(括弧)
|
||||
文字
|
||||
【括弧】
|
||||
日本語
|
||||
English
|
||||
|
||||
「禁則(きんそく)処理(しょり)!」
|
||||
「禁則(きんそく)処理(しょり)!」
|
||||
「禁則(きんそく)処理(しょり)!」
|
||||
「禁則(きんそく)処理(しょり)」
|
||||
「禁則(きんそく)処理(しょり)!」
|
||||
「禁則(きんそく)処理(しょり)!」
|
||||
「禁則(きんそく)処理(しょり)」
|
||||
「禁則(きんそく)処理(しょり)」
|
||||
「禁則(きんそく)処理(しょり)!!!!」
|
||||
「禁則(きんそく)処理(しょり)!!!」
|
||||
「禁則(きんそく)処理(しょり)!」
|
||||
「禁則(きんそく)処理(しょり)!」
|
||||
「禁則(きんそく)処理(しょり)!」
|
||||
「禁則(きんそく)処理(しょり)!」
|
||||
「禁則(きんそく)処理(しょり)!」
|
||||
「禁則(きんそく)処理(しょり)!」
|
||||
|
||||
中点
|
||||
・
|
||||
中点
|
||||
|
||||
禁則処理にわざと違反した文章のテストを今から行います。準備はいいでしょうか?レデ
|
||||
ィ、ゴー!
|
||||
|
||||
[ウ ィキペディア]
|
||||
|
||||
[ウ ィキペディア]: https://ja.wikipedia.org/
|
||||
|
||||
C言
|
||||
語
|
||||
・
|
||||
C++
|
||||
・
|
||||
Go
|
||||
・
|
||||
Rust
|
||||
|
||||
`````
|
||||
|
||||
### Expected (prettier)
|
||||
|
||||
`````md
|
||||
日本語、にほんご。汉语, 中文. 日本語,にほんご.English
|
||||
words!? 漢字!汉字?「セリフ」(括弧) 文字(括弧)文字【括弧】日本語English
|
||||
|
||||
「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)」「禁則(きんそく)処理(しょり)」「禁則(きんそく)処理(しょり)!!!!」「禁則(きんそく)処理(しょり)!!!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」「禁則(きんそく)処理(しょり)!」
|
||||
|
||||
中点・中点
|
||||
|
||||
禁則処理にわざと違反した文章のテストを今から行います。準備はいいでしょうか?レディ、ゴー!
|
||||
|
||||
[ウ ィキペディア]
|
||||
|
||||
[ウ ィキペディア]: https://ja.wikipedia.org/
|
||||
|
||||
C言語・C++・Go・Rust
|
||||
|
||||
`````
|
||||
@@ -87,7 +87,7 @@ Pin fixtures are named after their entry's slug (a comment would be an HTML bloc
|
||||
### Fixture fingerprint
|
||||
|
||||
`tests/fixtures/fingerprint.rs` serializes the AST's meaning (structure, decoded text, destinations, labels) and ignores what formatting may change
|
||||
(spans, markers, fence style, text splitting, whitespace runs, tightness, whitespace next to Chinese / Japanese characters, trailing whitespace of verbatim lines, info string whitespace).
|
||||
(spans, markers, fence style, text splitting, whitespace runs, tightness, trailing whitespace of verbatim lines, info string whitespace).
|
||||
The harness asserts it is identical for input and output: every fixture is a `parse(format(x)) ≅ parse(x)` check, which idempotency alone cannot give (a corrupted output is often a fixpoint).
|
||||
Extend the ignore set only with a reason written next to it.
|
||||
|
||||
|
||||
@@ -505,27 +505,42 @@ delimiter escaped when a later line could close it as front matter (a line start
|
||||
or a thematic break, which prints `---`): the next parse (Prettier's and ours) would read the block as front matter.
|
||||
Prettier prints it as written (the leading blank line that kept it out of front matter is dropped).
|
||||
|
||||
## autolink-cjk-space
|
||||
## cj-line-break
|
||||
|
||||
- Why: semantics
|
||||
- Pin: `tests/fixtures/markdown/prose-wrap/autolink-cjk-space.md`
|
||||
- Why: semantics (prettier/prettier#20143)
|
||||
- Pin: `tests/fixtures/markdown/prose-wrap/cj-line-break.md`
|
||||
- Conformance: `markdown/splitCjkText/symbolSpaceNewLine.md`
|
||||
- Oxfmt: `externals/prettier/markdown/splitCjkText/symbolSpaceNewLine.md`
|
||||
|
||||
```markdown
|
||||
<!-- input, proseWrap always -->
|
||||
日本語の文章は
|
||||
改行しても
|
||||
そのままです。
|
||||
|
||||
見て http://x.y2.
|
||||
。次
|
||||
|
||||
<!-- ours -->
|
||||
見て http://x.y2. 。次
|
||||
日本語の文章は
|
||||
改行しても
|
||||
そのままです。
|
||||
|
||||
見て http://x.y2.
|
||||
。次
|
||||
|
||||
<!-- prettier -->
|
||||
日本語の文章は改行してもそのままです。
|
||||
|
||||
見て http://x.y2.。次
|
||||
```
|
||||
|
||||
A line break right after the trailing punctuation of an autolink literal stays a space, whatever the
|
||||
Chinese / Japanese rules would make of it: the literal runs to the next whitespace, so joined to `。次`
|
||||
it becomes `http://x.y2.。次` on the next parse (`autolink_stretch`).
|
||||
Prettier drops the break between the two punctuation characters and the link changes.
|
||||
A line break next to Chinese / Japanese text is kept under every `proseWrap`,
|
||||
except between a Korean and a CJ letter, where it is a space.
|
||||
Browsers disagree on such a break (Firefox drops it, Chrome and Safari render a space),
|
||||
so removing it or making it a space changes the rendered text in some of them.
|
||||
Joined, an autolink literal also takes the CJ punctuation after it (`http://x.y2.。次`).
|
||||
Prettier `main` keeps the break since #20143; the pin (3.9.9) removes it or makes it a space.
|
||||
|
||||
## wiki-link-risk-link-text
|
||||
|
||||
|
||||
@@ -1,20 +1,18 @@
|
||||
//! Word kinds for the CJK line-breaking rules (Prettier's `splitText` / `printWhitespace`).
|
||||
//!
|
||||
//! A whitespace-delimited word is split into runs of non-CJK characters
|
||||
//! and single CJK characters (a variation selector stays with its base);
|
||||
//! only the edge runs matter to the printer,
|
||||
//! and the split as a whole feeds the "does this sentence put spaces around CJ text" statistic.
|
||||
//! Only the characters at the ends of a whitespace-delimited word matter to the printer
|
||||
//! (a variation selector stays with its base).
|
||||
//!
|
||||
//! The tables are Prettier's own character classes
|
||||
//! (`constants.evaluate.js`: `cjk-regex` + Script_Extensions / General_Category for CJK,
|
||||
//! ASCII punctuation + `\p{P}` (+ U+3000, U+FF5E) for punctuation, `\p{Script_Extensions=Hangul}` for Korean),
|
||||
//! `\p{P}` (+ U+3000, U+FF5E) for punctuation, `\p{Script_Extensions=Hangul}` for Korean),
|
||||
//! enumerated with Node 26 (Unicode 17).
|
||||
//! Regenerate by testing every code point against those regexes and collapsing the hits into ranges.
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum Kind {
|
||||
NonCjk,
|
||||
/// Chinese or Japanese letter: no spaces between words, a line break is not a space.
|
||||
/// Chinese or Japanese letter: no spaces between words, a line break next to it is kept.
|
||||
CjLetter,
|
||||
/// Korean letter: words are space-separated, as in Latin script.
|
||||
KLetter,
|
||||
@@ -27,67 +25,33 @@ impl Kind {
|
||||
}
|
||||
}
|
||||
|
||||
/// The run at one end of a word, as the whitespace next to it sees it.
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct Edge {
|
||||
pub kind: Kind,
|
||||
/// The run's character on the whitespace side.
|
||||
pub ch: char,
|
||||
/// That character is punctuation (a CJK punctuation run always is).
|
||||
pub punctuation: bool,
|
||||
/// The kind of the word's first character.
|
||||
pub fn first_kind(word: &str) -> Option<Kind> {
|
||||
word.chars().next().map(kind)
|
||||
}
|
||||
|
||||
/// The kinds of a word's runs, in order.
|
||||
pub fn kinds(word: &str) -> impl Iterator<Item = Kind> + '_ {
|
||||
runs(word).map(|(_, kind)| kind)
|
||||
}
|
||||
|
||||
/// The word's first and last run, as edges.
|
||||
pub fn edges(word: &str) -> Option<(Edge, Edge)> {
|
||||
let mut it = runs(word);
|
||||
let (first_run, first_kind) = it.next()?;
|
||||
let (last_run, last_kind) = it.last().unwrap_or((first_run, first_kind));
|
||||
let first_ch = first_run.chars().next()?;
|
||||
let last_ch = last_run.chars().next_back()?;
|
||||
let edge = |kind: Kind, ch: char| Edge {
|
||||
kind,
|
||||
ch,
|
||||
punctuation: kind == Kind::CjkPunctuation || is_punctuation(ch),
|
||||
};
|
||||
Some((edge(first_kind, first_ch), edge(last_kind, last_ch)))
|
||||
}
|
||||
|
||||
/// Splits a word into runs: each CJK character (plus a following variation selector) on its own,
|
||||
/// non-CJK characters together.
|
||||
fn runs(word: &str) -> impl Iterator<Item = (&str, Kind)> {
|
||||
let mut rest = word;
|
||||
std::iter::from_fn(move || {
|
||||
let c = rest.chars().next()?;
|
||||
let (len, kind) = if is_cjk(c) {
|
||||
let mut len = c.len_utf8();
|
||||
if let Some(vs) = rest[len..].chars().next()
|
||||
&& is_variation_selector(vs)
|
||||
{
|
||||
len += vs.len_utf8();
|
||||
}
|
||||
let kind = if is_punctuation(c) {
|
||||
Kind::CjkPunctuation
|
||||
} else if is_hangul(c) {
|
||||
Kind::KLetter
|
||||
} else {
|
||||
Kind::CjLetter
|
||||
};
|
||||
(len, kind)
|
||||
} else {
|
||||
let len = rest.find(is_cjk).unwrap_or(rest.len());
|
||||
(len, Kind::NonCjk)
|
||||
};
|
||||
let (run, tail) = rest.split_at(len);
|
||||
rest = tail;
|
||||
Some((run, kind))
|
||||
/// The kind of the word's last character; a trailing variation selector stays with its base.
|
||||
pub fn last_kind(word: &str) -> Option<Kind> {
|
||||
let mut chars = word.chars();
|
||||
let last = chars.next_back()?;
|
||||
Some(match chars.next_back() {
|
||||
Some(base) if is_variation_selector(last) && is_cjk(base) => kind(base),
|
||||
_ => kind(last),
|
||||
})
|
||||
}
|
||||
|
||||
fn kind(c: char) -> Kind {
|
||||
if !is_cjk(c) {
|
||||
Kind::NonCjk
|
||||
} else if is_punctuation(c) {
|
||||
Kind::CjkPunctuation
|
||||
} else if is_hangul(c) {
|
||||
Kind::KLetter
|
||||
} else {
|
||||
Kind::CjLetter
|
||||
}
|
||||
}
|
||||
|
||||
fn is_variation_selector(c: char) -> bool {
|
||||
matches!(u32::from(c), 0xFE00..=0xFE0F | 0xE0100..=0xE01EF)
|
||||
}
|
||||
@@ -100,11 +64,8 @@ fn is_hangul(c: char) -> bool {
|
||||
in_ranges(HANGUL, c)
|
||||
}
|
||||
|
||||
/// ASCII punctuation, U+3000, U+FF5E, or Unicode general category P.
|
||||
pub fn is_punctuation(c: char) -> bool {
|
||||
if c.is_ascii() {
|
||||
return c.is_ascii_punctuation();
|
||||
}
|
||||
/// U+3000, U+FF5E, or Unicode general category P (only asked about CJK characters).
|
||||
fn is_punctuation(c: char) -> bool {
|
||||
matches!(c, '\u{3000}' | '\u{FF5E}') || in_ranges(PUNCTUATION, c)
|
||||
}
|
||||
|
||||
|
||||
@@ -272,13 +272,13 @@ pub fn collect_inlines<'a>(
|
||||
let break_kept = |j: usize, f: &MarkdownFormatter<'_, 'a>| -> bool {
|
||||
f.context().raw_text().get() != Raw::No || raw_line(j + 1, f) != Raw::No
|
||||
};
|
||||
// Prettier's sentence is a run of texts joined by soft breaks; its CJ spacing style is one statistic.
|
||||
let mut sentence_cj_spaces: Option<Option<bool>> = None;
|
||||
// Whether the sentence (a run of texts joined by soft breaks) has CJK text, scanned once per sentence.
|
||||
let mut sentence_has_cjk: Option<bool> = None;
|
||||
// The previous sibling was a line break that stays in the output
|
||||
let mut last_break_kept = false;
|
||||
for (i, child) in children.iter().enumerate() {
|
||||
if !matches!(child, Inline::Text(_) | Inline::SoftBreak(_)) {
|
||||
sentence_cj_spaces = None;
|
||||
sentence_has_cjk = None;
|
||||
}
|
||||
let after_kept_break = last_break_kept;
|
||||
if !matches!(child, Inline::SoftBreak(_) | Inline::HardBreak(_)) {
|
||||
@@ -332,8 +332,8 @@ pub fn collect_inlines<'a>(
|
||||
} else {
|
||||
(None, None)
|
||||
};
|
||||
let cj_spaces = *sentence_cj_spaces
|
||||
.get_or_insert_with(|| sentence_cj_spaces_at(children, i, f));
|
||||
let has_cjk =
|
||||
*sentence_has_cjk.get_or_insert_with(|| sentence_has_cjk_at(children, i));
|
||||
// A paragraph's first line that opens a block
|
||||
// (the rest of a paragraph a definition was split from) is escaped: `\- x`, `1\. x`
|
||||
if i == 0
|
||||
@@ -365,7 +365,7 @@ pub fn collect_inlines<'a>(
|
||||
before_soft_break: matches!(children.get(i + 1), Some(Inline::SoftBreak(_))),
|
||||
next_word: next_word_of(children, i, parent.delimiter, f),
|
||||
glued_last_word: glued_last_word(children, i, raw, f),
|
||||
cj_spaces,
|
||||
has_cjk,
|
||||
..words::TextContext::default()
|
||||
};
|
||||
words::push_text(raw, cx, parts, f);
|
||||
@@ -381,19 +381,19 @@ pub fn collect_inlines<'a>(
|
||||
// A separator, not content: the next line is measured on its own
|
||||
parts.push_sep(Sep::HardLine);
|
||||
} else {
|
||||
let cj_spaces = *sentence_cj_spaces
|
||||
.get_or_insert_with(|| sentence_cj_spaces_at(children, i, f));
|
||||
let has_cjk =
|
||||
*sentence_has_cjk.get_or_insert_with(|| sentence_has_cjk_at(children, i));
|
||||
let cx = words::TextContext {
|
||||
next_word: next_word_of(children, i, parent.delimiter, f),
|
||||
after_liquid: follows_liquid(children, i),
|
||||
// Only the CJK rules look back (a trailing `\` was escaped by the text);
|
||||
// a word glued to an autolink literal is part of it, the break after it stays a space
|
||||
prev_word: if cj_spaces.is_some() && !glued_to_autolink(children, i, f) {
|
||||
// Only the CJK rules look back (a trailing `\` was escaped by the text),
|
||||
// and `preserve` keeps the break anyway
|
||||
prev_word: if has_cjk && f.options().prose_wrap != ProseWrap::Preserve {
|
||||
prev_word_of(children, i, f)
|
||||
} else {
|
||||
None
|
||||
},
|
||||
cj_spaces,
|
||||
has_cjk,
|
||||
..words::TextContext::default()
|
||||
};
|
||||
words::push_whitespace(true, &cx, parts, f);
|
||||
@@ -707,40 +707,17 @@ fn autolink_stretch(children: &[Inline<'_>], i: usize, raw: &str) -> usize {
|
||||
}
|
||||
}
|
||||
|
||||
/// The text before node `i` is one word in an autolink literal's stretch (`http://x.y2` + `.`):
|
||||
/// what follows the break must not glue to it (`push_text` makes the same call for its first word).
|
||||
fn glued_to_autolink<'a>(
|
||||
children: &'a [Inline<'a>],
|
||||
i: usize,
|
||||
f: &MarkdownFormatter<'_, 'a>,
|
||||
) -> bool {
|
||||
let Some(Inline::Text(t)) = children.get(i.wrapping_sub(1)) else { return false };
|
||||
let raw = f.context().slice(t.span);
|
||||
!raw.contains(is_split_whitespace) && autolink_stretch(children, i - 1, raw) > 0
|
||||
}
|
||||
|
||||
fn follows_liquid(children: &[Inline<'_>], i: usize) -> bool {
|
||||
matches!(children.get(i.wrapping_sub(1)), Some(Inline::Liquid(_)))
|
||||
}
|
||||
|
||||
/// `usesCJSpaces` of the sentence (run of texts and soft breaks) containing node `i`;
|
||||
/// `None` without CJK text, which is the common case and skips the scan and the CJK rules.
|
||||
fn sentence_cj_spaces_at<'a>(
|
||||
children: &'a [Inline<'a>],
|
||||
i: usize,
|
||||
f: &MarkdownFormatter<'_, 'a>,
|
||||
) -> Option<bool> {
|
||||
let in_sentence = |c: &Inline<'a>| matches!(c, Inline::Text(_) | Inline::SoftBreak(_));
|
||||
/// The sentence (run of texts and soft breaks) containing node `i` has CJK text;
|
||||
/// without it, the common case, the CJK rules are skipped.
|
||||
fn sentence_has_cjk_at(children: &[Inline<'_>], i: usize) -> bool {
|
||||
let in_sentence = |c: &Inline<'_>| matches!(c, Inline::Text(_) | Inline::SoftBreak(_));
|
||||
let start = (0..i).rev().take_while(|&j| in_sentence(&children[j])).last().unwrap_or(i);
|
||||
let end = (i..children.len()).take_while(|&j| in_sentence(&children[j])).last().unwrap_or(i);
|
||||
let texts = children[start..=end].iter().filter_map(|c| match c {
|
||||
Inline::Text(t) => Some(t),
|
||||
_ => None,
|
||||
});
|
||||
if !texts.clone().any(|t| t.contains_cjk) {
|
||||
return None;
|
||||
}
|
||||
Some(words::uses_cj_spaces(texts.map(|t| f.context().slice(t.span))))
|
||||
children[start..=end].iter().any(|c| matches!(c, Inline::Text(t) if t.contains_cjk))
|
||||
}
|
||||
|
||||
/// A word character directly before or after node `i`.
|
||||
|
||||
@@ -5,7 +5,7 @@ use oxc_allocator::{Allocator, ArenaVec, StringBuilder};
|
||||
use oxc_formatter_core::{
|
||||
Buffer, Format,
|
||||
builders::{
|
||||
dedent_to_root, hard_line_break, literal_line_break, mark_as_root, soft_line_break,
|
||||
dedent_to_root, hard_line_break, literal_line_break, mark_as_root,
|
||||
soft_line_break_or_space, text,
|
||||
},
|
||||
write,
|
||||
@@ -46,8 +46,6 @@ impl<'a> Format<'a, MarkdownFormatContext<'a>> for Atom<'a> {
|
||||
/// A fill separator.
|
||||
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
|
||||
pub enum Sep {
|
||||
/// A break or nothing (a line break between CJ characters).
|
||||
SoftLine,
|
||||
Line,
|
||||
HardLine,
|
||||
}
|
||||
@@ -55,7 +53,6 @@ pub enum Sep {
|
||||
impl<'a> Format<'a, MarkdownFormatContext<'a>> for Sep {
|
||||
fn fmt(&self, f: &mut MarkdownFormatter<'_, 'a>) {
|
||||
match self {
|
||||
Sep::SoftLine => write!(f, soft_line_break()),
|
||||
Sep::Line => write!(f, soft_line_break_or_space()),
|
||||
Sep::HardLine => write!(f, hard_line_break()),
|
||||
}
|
||||
@@ -79,13 +76,12 @@ pub struct Parts<'a> {
|
||||
}
|
||||
|
||||
impl<'a> Item<'a> {
|
||||
/// The item as the parser will read it: a break is a newline, a soft line between CJ letters nothing.
|
||||
/// The item as the parser will read it: a break is a newline, a line a space.
|
||||
fn flat(&self) -> &'a str {
|
||||
match self {
|
||||
Item::Atom(Atom::Str(s)) => s,
|
||||
Item::Atom(_) | Item::Sep(Sep::HardLine) => "\n",
|
||||
Item::Sep(Sep::Line) => " ",
|
||||
Item::Sep(Sep::SoftLine) => "",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
//! - a line break before a word that would start a block (`-`, `1.`, `#`, `>`) never breaks
|
||||
//!
|
||||
//! Around Chinese / Japanese text a whitespace follows Prettier's `printWhitespace`:
|
||||
//! a line break between CJ characters is not a space, and a space next to one never breaks
|
||||
//! a line break next to CJ is kept, and a space next to one never breaks
|
||||
//! (`cjk` classifies the word edges).
|
||||
|
||||
use std::borrow::Cow;
|
||||
@@ -19,7 +19,7 @@ use crate::options::ProseWrap;
|
||||
|
||||
use super::{
|
||||
MarkdownFormatter,
|
||||
cjk::{self, Edge, Kind},
|
||||
cjk::{self, Kind},
|
||||
escape::{
|
||||
ends_with_unescaped_backslash, is_fake_setext_underline, leading_run_escaped,
|
||||
print_delimited_word,
|
||||
@@ -76,9 +76,8 @@ pub struct TextContext<'a> {
|
||||
/// The last word of the previous sibling text (a soft break's other side), for the CJK rules only;
|
||||
/// a trailing `\` there was escaped by the text (see `push_text`).
|
||||
pub prev_word: Option<&'a str>,
|
||||
/// The sentence puts spaces between CJ and non-CJK words (Prettier's `usesCJSpaces`);
|
||||
/// `None` when it has no CJK text at all.
|
||||
pub cj_spaces: Option<bool>,
|
||||
/// The sentence has CJK text (the parser's `contains_cjk`); without it the word edges are not classified.
|
||||
pub has_cjk: bool,
|
||||
/// Inside a reference link's raw content, where a line break never prevents a break.
|
||||
pub is_link: bool,
|
||||
}
|
||||
@@ -99,8 +98,6 @@ pub fn push_text<'a>(
|
||||
let mut is_first = true;
|
||||
let leading_ws = raw.starts_with(is_split_whitespace);
|
||||
let mut prev_word: Option<&'a str> = None;
|
||||
// `prev_word` is the text's first word
|
||||
let mut prev_is_first = false;
|
||||
loop {
|
||||
// Whitespace before the next word (leading, or the run after the previous word).
|
||||
let after_ws = rest.trim_start_matches(is_split_whitespace);
|
||||
@@ -126,9 +123,7 @@ pub fn push_text<'a>(
|
||||
};
|
||||
let cx = TextContext {
|
||||
next_word: next,
|
||||
// The first word inside an autolink literal's stretch is part of the link:
|
||||
// the whitespace after it stays a space (a CJ neighbor must not glue to the link)
|
||||
prev_word: prev_word.filter(|_| !(cx.autolink_stretch > 0 && prev_is_first)),
|
||||
prev_word,
|
||||
after_liquid: is_first && cx.after_liquid,
|
||||
..cx
|
||||
};
|
||||
@@ -144,7 +139,6 @@ pub fn push_text<'a>(
|
||||
break;
|
||||
}
|
||||
prev_word = Some(word);
|
||||
prev_is_first = is_first;
|
||||
let printed: Cow<'a, str> = if in_delimiter {
|
||||
let prev = if is_first { cx.edge_prev } else { Some(' ') };
|
||||
let next = if is_last { cx.edge_next } else { Some(' ') };
|
||||
@@ -184,7 +178,7 @@ pub fn push_text<'a>(
|
||||
|
||||
/// The whitespace (a space run, or one holding a `newline`) between `cx.prev_word`
|
||||
/// and `cx.next_word` (`None`: the whitespace touches a node edge):
|
||||
/// a separator that may break, a space, or nothing.
|
||||
/// a kept line break, a separator that may break, or a space.
|
||||
pub fn push_whitespace<'a>(
|
||||
newline: bool,
|
||||
cx: &TextContext<'a>,
|
||||
@@ -208,134 +202,44 @@ pub fn push_whitespace<'a>(
|
||||
options.prose_wrap
|
||||
};
|
||||
|
||||
if prose_wrap == ProseWrap::Preserve && newline {
|
||||
if newline && prose_wrap == ProseWrap::Preserve {
|
||||
parts.push_sep(Sep::HardLine);
|
||||
return;
|
||||
}
|
||||
// The CJK rules only apply to a sentence with CJK text
|
||||
let (prev, next) = if cx.cj_spaces.is_some() {
|
||||
// A space breaks only under `always`; otherwise it stays a space whatever its neighbors
|
||||
let (prev, next) = if cx.has_cjk && (newline || prose_wrap == ProseWrap::Always) {
|
||||
(
|
||||
cx.prev_word.and_then(|w| cjk::edges(w).map(|(_, last)| last)),
|
||||
cx.next_word
|
||||
.filter(|w| w.in_sentence)
|
||||
.and_then(|w| cjk::edges(w.word).map(|(first, _)| first)),
|
||||
cx.prev_word.and_then(cjk::last_kind),
|
||||
cx.next_word.filter(|w| w.in_sentence).and_then(|w| cjk::first_kind(w.word)),
|
||||
)
|
||||
} else {
|
||||
(None, None)
|
||||
};
|
||||
let can_be_space = !newline || line_break_can_be_space(prev, next, cx);
|
||||
// Next to CJ, except Korean next to a CJ letter (a space everywhere)
|
||||
let cj_bound = (prev.is_some_and(Kind::is_cj) || next.is_some_and(Kind::is_cj))
|
||||
&& !matches!(
|
||||
(prev, next),
|
||||
(Some(Kind::KLetter), Some(Kind::CjLetter))
|
||||
| (Some(Kind::CjLetter), Some(Kind::KLetter))
|
||||
);
|
||||
// Prettier's `isLineBreakAmbiguous`: browsers disagree on whether the break is a space or nothing
|
||||
if newline && cj_bound {
|
||||
parts.push_sep(Sep::HardLine);
|
||||
return;
|
||||
}
|
||||
// Prettier's `isBreakable`: never a break between CJ and a neighbor word,
|
||||
// a node edge (`None`) may break
|
||||
let breakable = prose_wrap == ProseWrap::Always
|
||||
&& f.context().no_wrap_depth().get() == 0
|
||||
&& is_breakable(prev, next, cx);
|
||||
match (breakable, can_be_space) {
|
||||
(true, true) => parts.push_sep(Sep::Line),
|
||||
(true, false) => parts.push_sep(Sep::SoftLine),
|
||||
(false, true) => parts.push_str(" "),
|
||||
(false, false) => {}
|
||||
&& !(cj_bound && prev.is_some() && next.is_some());
|
||||
// Any line break left here is a space (Prettier's `lineBreakCanBeConvertedToSpace`)
|
||||
if breakable {
|
||||
parts.push_sep(Sep::Line);
|
||||
} else {
|
||||
parts.push_str(" ");
|
||||
}
|
||||
}
|
||||
|
||||
/// Prettier's `lineBreakCanBeConvertedToSpace`.
|
||||
fn line_break_can_be_space(prev: Option<Edge>, next: Option<Edge>, cx: &TextContext<'_>) -> bool {
|
||||
if cx.is_link {
|
||||
return true;
|
||||
}
|
||||
let (Some(prev), Some(next)) = (prev, next) else { return true };
|
||||
let latin_like = |kind: Kind| matches!(kind, Kind::NonCjk | Kind::KLetter);
|
||||
// Between non-CJK / Korean words, or Korean and CJ, a line break is a space
|
||||
if (latin_like(prev.kind) && latin_like(next.kind))
|
||||
|| matches!(
|
||||
(prev.kind, next.kind),
|
||||
(Kind::KLetter, Kind::CjLetter) | (Kind::CjLetter, Kind::KLetter)
|
||||
)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
// A delimiter run glued to a CJ character gains flanking it did not have (`」\n**` → `」**` closes);
|
||||
// a space keeps it inert, as the line break did
|
||||
if is_delimiter_char(prev.ch) || is_delimiter_char(next.ch) {
|
||||
return true;
|
||||
}
|
||||
// Around CJK punctuation or between CJ letters it is nothing
|
||||
if prev.kind == Kind::CjkPunctuation
|
||||
|| next.kind == Kind::CjkPunctuation
|
||||
|| (prev.kind == Kind::CjLetter && next.kind == Kind::CjLetter)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
// Between CJ and non-CJK: a space next to ASCII punctuation (`:::` fences),
|
||||
// nothing next to other punctuation (`〜`, `…`), else the sentence's own style.
|
||||
if prev.ch.is_ascii_punctuation() || next.ch.is_ascii_punctuation() {
|
||||
return true;
|
||||
}
|
||||
if prev.punctuation || next.punctuation {
|
||||
return false;
|
||||
}
|
||||
cx.cj_spaces.unwrap_or(false)
|
||||
}
|
||||
|
||||
/// `*` `_` `~` `` ` ``: runs of these pair up by flanking or by length, so their neighbors matter.
|
||||
fn is_delimiter_char(c: char) -> bool {
|
||||
matches!(c, '*' | '_' | '~' | '`')
|
||||
}
|
||||
|
||||
/// Prettier's `isBreakable` for a `" "` / `"\n"` whitespace under `always`.
|
||||
fn is_breakable(prev: Option<Edge>, next: Option<Edge>, cx: &TextContext<'_>) -> bool {
|
||||
if cx.is_link {
|
||||
return true;
|
||||
}
|
||||
let (Some(prev), Some(next)) = (prev, next) else { return true };
|
||||
// Korean next to CJ breaks;
|
||||
// anything else next to CJ never does (browsers turn the break into a space)
|
||||
matches!(
|
||||
(prev.kind, next.kind),
|
||||
(Kind::KLetter, Kind::CjLetter) | (Kind::CjLetter, Kind::KLetter)
|
||||
) || !(prev.kind.is_cj() || next.kind.is_cj())
|
||||
}
|
||||
|
||||
/// Prettier's `usesCJSpaces`, over the words of one sentence (the texts of a soft-break run):
|
||||
/// whether spaces outnumber nothing between CJ and non-CJK words.
|
||||
/// `words` yields each text's words with the whitespace before them (`None` for the first).
|
||||
pub fn uses_cj_spaces<'a>(texts: impl Iterator<Item = &'a str>) -> bool {
|
||||
let (mut spaces, mut nothing) = (0u32, 0u32);
|
||||
let counts = |a: Kind, b: Kind| {
|
||||
matches!((a, b), (Kind::CjLetter, Kind::NonCjk) | (Kind::NonCjk, Kind::CjLetter))
|
||||
};
|
||||
for text in texts {
|
||||
let mut prev_last: Option<Kind> = None;
|
||||
let mut rest = text;
|
||||
loop {
|
||||
let after_ws = rest.trim_start_matches(is_split_whitespace);
|
||||
let ws = &rest[..rest.len() - after_ws.len()];
|
||||
rest = after_ws;
|
||||
let word_end = rest.find(is_split_whitespace).unwrap_or(rest.len());
|
||||
let (word, tail) = rest.split_at(word_end);
|
||||
if word.is_empty() {
|
||||
break;
|
||||
}
|
||||
let mut kinds = cjk::kinds(word);
|
||||
let first = kinds.next().unwrap_or(Kind::NonCjk);
|
||||
if let Some(prev) = prev_last
|
||||
&& !ws.is_empty()
|
||||
&& !ws.contains('\n')
|
||||
&& counts(prev, first)
|
||||
{
|
||||
spaces += 1;
|
||||
}
|
||||
let mut last = first;
|
||||
for kind in kinds {
|
||||
if counts(last, kind) {
|
||||
nothing += 1;
|
||||
}
|
||||
last = kind;
|
||||
}
|
||||
prev_last = Some(last);
|
||||
rest = tail;
|
||||
}
|
||||
}
|
||||
spaces > nothing
|
||||
}
|
||||
|
||||
/// Never break before a word that would start a block.
|
||||
pub fn prevents_break(newline: bool, next: Option<NextWord<'_>>, prose_wrap: ProseWrap) -> bool {
|
||||
let Some(next) = next else { return false };
|
||||
|
||||
+1
-28
@@ -312,33 +312,6 @@ impl Walker<'_> {
|
||||
}
|
||||
|
||||
/// Whitespace runs are a formatter freedom (`proseWrap`, indentation), their content is not.
|
||||
/// A run touching a Chinese / Japanese character is nothing: the printer treats a space
|
||||
/// and a line break next to CJ text as interchangeable with no whitespace
|
||||
/// (browsers drop a segment break between CJ characters), so the oracle equates all three.
|
||||
fn collapse_whitespace(s: &str) -> String {
|
||||
let mut out = String::with_capacity(s.len());
|
||||
let mut pending_space = false;
|
||||
for word in s.split_whitespace() {
|
||||
if pending_space && !out.ends_with(is_cj) && !word.starts_with(is_cj) {
|
||||
out.push(' ');
|
||||
}
|
||||
out.push_str(word);
|
||||
pending_space = true;
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Chinese / Japanese (not Korean) by Unicode block; deliberately not the printer's table.
|
||||
fn is_cj(c: char) -> bool {
|
||||
matches!(u32::from(c),
|
||||
0x2E80..=0x303F // CJK radicals, Kangxi, ideographic description, CJK symbols and punctuation
|
||||
| 0x3040..=0x30FF // Hiragana, Katakana
|
||||
| 0x3100..=0x312F // Bopomofo
|
||||
| 0x31C0..=0x31EF // CJK strokes
|
||||
| 0x3200..=0x9FFF // enclosed CJK, compatibility, ext A, unified ideographs
|
||||
| 0xF900..=0xFAFF // compatibility ideographs
|
||||
| 0xFE30..=0xFE4F // compatibility forms
|
||||
| 0xFF00..=0xFF9F // fullwidth forms, halfwidth katakana
|
||||
| 0x20000..=0x3FFFF // ext B and beyond
|
||||
)
|
||||
s.split_whitespace().collect::<Vec<_>>().join(" ")
|
||||
}
|
||||
-2
@@ -1,2 +0,0 @@
|
||||
見て http://x.y2.
|
||||
。次
|
||||
Vendored
-41
@@ -1,41 +0,0 @@
|
||||
---
|
||||
source: crates/oxc_formatter_markdown/tests/fixtures/mod.rs
|
||||
---
|
||||
==================== Input ====================
|
||||
見て http://x.y2.
|
||||
。次
|
||||
|
||||
==================== Output ====================
|
||||
---------------------------------------
|
||||
{ printWidth: 80, proseWrap: "always" }
|
||||
---------------------------------------
|
||||
見て http://x.y2. 。次
|
||||
|
||||
----------------------------------------
|
||||
{ printWidth: 100, proseWrap: "always" }
|
||||
----------------------------------------
|
||||
見て http://x.y2. 。次
|
||||
|
||||
--------------------------------------
|
||||
{ printWidth: 80, proseWrap: "never" }
|
||||
--------------------------------------
|
||||
見て http://x.y2. 。次
|
||||
|
||||
---------------------------------------
|
||||
{ printWidth: 100, proseWrap: "never" }
|
||||
---------------------------------------
|
||||
見て http://x.y2. 。次
|
||||
|
||||
-----------------------------------------
|
||||
{ printWidth: 80, proseWrap: "preserve" }
|
||||
-----------------------------------------
|
||||
見て http://x.y2.
|
||||
。次
|
||||
|
||||
------------------------------------------
|
||||
{ printWidth: 100, proseWrap: "preserve" }
|
||||
------------------------------------------
|
||||
見て http://x.y2.
|
||||
。次
|
||||
|
||||
===================== End =====================
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
日本語の文章は
|
||||
改行しても
|
||||
そのままです。
|
||||
|
||||
漢字と English の間の改行も
|
||||
English の前で残ります。
|
||||
|
||||
営業時間は月〜
|
||||
金の9時〜18時です。
|
||||
|
||||
中文!
|
||||
한국어
|
||||
|
||||
한국어와
|
||||
漢字の間の改行は空白になります。
|
||||
|
||||
更多细节,请参考文档:
|
||||
www.example.com 和 `code`。
|
||||
|
||||
見て http://x.y2.
|
||||
。次
|
||||
+174
@@ -0,0 +1,174 @@
|
||||
---
|
||||
source: crates/oxc_formatter_markdown/tests/fixtures/mod.rs
|
||||
---
|
||||
==================== Input ====================
|
||||
日本語の文章は
|
||||
改行しても
|
||||
そのままです。
|
||||
|
||||
漢字と English の間の改行も
|
||||
English の前で残ります。
|
||||
|
||||
営業時間は月〜
|
||||
金の9時〜18時です。
|
||||
|
||||
中文!
|
||||
한국어
|
||||
|
||||
한국어와
|
||||
漢字の間の改行は空白になります。
|
||||
|
||||
更多细节,请参考文档:
|
||||
www.example.com 和 `code`。
|
||||
|
||||
見て http://x.y2.
|
||||
。次
|
||||
|
||||
==================== Output ====================
|
||||
---------------------------------------
|
||||
{ printWidth: 80, proseWrap: "always" }
|
||||
---------------------------------------
|
||||
日本語の文章は
|
||||
改行しても
|
||||
そのままです。
|
||||
|
||||
漢字と English の間の改行も
|
||||
English の前で残ります。
|
||||
|
||||
営業時間は月〜
|
||||
金の9時〜18時です。
|
||||
|
||||
中文!
|
||||
한국어
|
||||
|
||||
한국어와 漢字の間の改行は空白になります。
|
||||
|
||||
更多细节,请参考文档:
|
||||
www.example.com 和 `code`。
|
||||
|
||||
見て http://x.y2.
|
||||
。次
|
||||
|
||||
----------------------------------------
|
||||
{ printWidth: 100, proseWrap: "always" }
|
||||
----------------------------------------
|
||||
日本語の文章は
|
||||
改行しても
|
||||
そのままです。
|
||||
|
||||
漢字と English の間の改行も
|
||||
English の前で残ります。
|
||||
|
||||
営業時間は月〜
|
||||
金の9時〜18時です。
|
||||
|
||||
中文!
|
||||
한국어
|
||||
|
||||
한국어와 漢字の間の改行は空白になります。
|
||||
|
||||
更多细节,请参考文档:
|
||||
www.example.com 和 `code`。
|
||||
|
||||
見て http://x.y2.
|
||||
。次
|
||||
|
||||
--------------------------------------
|
||||
{ printWidth: 80, proseWrap: "never" }
|
||||
--------------------------------------
|
||||
日本語の文章は
|
||||
改行しても
|
||||
そのままです。
|
||||
|
||||
漢字と English の間の改行も
|
||||
English の前で残ります。
|
||||
|
||||
営業時間は月〜
|
||||
金の9時〜18時です。
|
||||
|
||||
中文!
|
||||
한국어
|
||||
|
||||
한국어와 漢字の間の改行は空白になります。
|
||||
|
||||
更多细节,请参考文档:
|
||||
www.example.com 和 `code`。
|
||||
|
||||
見て http://x.y2.
|
||||
。次
|
||||
|
||||
---------------------------------------
|
||||
{ printWidth: 100, proseWrap: "never" }
|
||||
---------------------------------------
|
||||
日本語の文章は
|
||||
改行しても
|
||||
そのままです。
|
||||
|
||||
漢字と English の間の改行も
|
||||
English の前で残ります。
|
||||
|
||||
営業時間は月〜
|
||||
金の9時〜18時です。
|
||||
|
||||
中文!
|
||||
한국어
|
||||
|
||||
한국어와 漢字の間の改行は空白になります。
|
||||
|
||||
更多细节,请参考文档:
|
||||
www.example.com 和 `code`。
|
||||
|
||||
見て http://x.y2.
|
||||
。次
|
||||
|
||||
-----------------------------------------
|
||||
{ printWidth: 80, proseWrap: "preserve" }
|
||||
-----------------------------------------
|
||||
日本語の文章は
|
||||
改行しても
|
||||
そのままです。
|
||||
|
||||
漢字と English の間の改行も
|
||||
English の前で残ります。
|
||||
|
||||
営業時間は月〜
|
||||
金の9時〜18時です。
|
||||
|
||||
中文!
|
||||
한국어
|
||||
|
||||
한국어와
|
||||
漢字の間の改行は空白になります。
|
||||
|
||||
更多细节,请参考文档:
|
||||
www.example.com 和 `code`。
|
||||
|
||||
見て http://x.y2.
|
||||
。次
|
||||
|
||||
------------------------------------------
|
||||
{ printWidth: 100, proseWrap: "preserve" }
|
||||
------------------------------------------
|
||||
日本語の文章は
|
||||
改行しても
|
||||
そのままです。
|
||||
|
||||
漢字と English の間の改行も
|
||||
English の前で残ります。
|
||||
|
||||
営業時間は月〜
|
||||
金の9時〜18時です。
|
||||
|
||||
中文!
|
||||
한국어
|
||||
|
||||
한국어와
|
||||
漢字の間の改行は空白になります。
|
||||
|
||||
更多细节,请参考文档:
|
||||
www.example.com 和 `code`。
|
||||
|
||||
見て http://x.y2.
|
||||
。次
|
||||
|
||||
===================== End =====================
|
||||
@@ -1,32 +1,11 @@
|
||||
<!-- Whitespace around Chinese / Japanese: a line break between CJ characters is nothing,
|
||||
a space next to CJ text never breaks, Korean behaves like Latin -->
|
||||
<!-- Whitespace around Chinese / Japanese: a space next to CJ text never breaks, Korean behaves like Latin
|
||||
(a line break next to CJ text: DIVERGENCES.md#cj-line-break) -->
|
||||
|
||||
日本語の文章は
|
||||
改行しても
|
||||
空白になりません。
|
||||
|
||||
漢字と English の間に空白を置く文書では、改行も
|
||||
English の前で空白になります。
|
||||
|
||||
漢字とEnglishの間に空白を置かない文書では、改行は
|
||||
Englishの前でも消えます。
|
||||
漢字と English の間に空白を置く文書でも、その空白では折り返さずに、段落の終わりまで一行のまま続きます。
|
||||
|
||||
한국어는 라틴 문자처럼
|
||||
띄어쓰기로 단어를 나눕니다.
|
||||
|
||||
営業時間は月〜
|
||||
金の9時〜
|
||||
18時です。詳しくは……
|
||||
後日お知らせします……
|
||||
|
||||
括弧の前 (paren) と後ろ、ASCII 記号の
|
||||
:: 前後は空白になります。
|
||||
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does:
|
||||
the whitespace there may break and a line break stays a space
|
||||
(joined, `:www.` would no longer be an autolink literal) -->
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does: the whitespace there may break -->
|
||||
|
||||
使用方法请参考文档中的说明,如果需要了解更多信息,请参考 [应用部署](./deployment.md)。
|
||||
|
||||
更多细节,请参考文档:
|
||||
www.example.com 和 `code`。
|
||||
+28
-139
@@ -2,215 +2,104 @@
|
||||
source: crates/oxc_formatter_markdown/tests/fixtures/mod.rs
|
||||
---
|
||||
==================== Input ====================
|
||||
<!-- Whitespace around Chinese / Japanese: a line break between CJ characters is nothing,
|
||||
a space next to CJ text never breaks, Korean behaves like Latin -->
|
||||
<!-- Whitespace around Chinese / Japanese: a space next to CJ text never breaks, Korean behaves like Latin
|
||||
(a line break next to CJ text: DIVERGENCES.md#cj-line-break) -->
|
||||
|
||||
日本語の文章は
|
||||
改行しても
|
||||
空白になりません。
|
||||
|
||||
漢字と English の間に空白を置く文書では、改行も
|
||||
English の前で空白になります。
|
||||
|
||||
漢字とEnglishの間に空白を置かない文書では、改行は
|
||||
Englishの前でも消えます。
|
||||
漢字と English の間に空白を置く文書でも、その空白では折り返さずに、段落の終わりまで一行のまま続きます。
|
||||
|
||||
한국어는 라틴 문자처럼
|
||||
띄어쓰기로 단어를 나눕니다.
|
||||
|
||||
営業時間は月〜
|
||||
金の9時〜
|
||||
18時です。詳しくは……
|
||||
後日お知らせします……
|
||||
|
||||
括弧の前 (paren) と後ろ、ASCII 記号の
|
||||
:: 前後は空白になります。
|
||||
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does:
|
||||
the whitespace there may break and a line break stays a space
|
||||
(joined, `:www.` would no longer be an autolink literal) -->
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does: the whitespace there may break -->
|
||||
|
||||
使用方法请参考文档中的说明,如果需要了解更多信息,请参考 [应用部署](./deployment.md)。
|
||||
|
||||
更多细节,请参考文档:
|
||||
www.example.com 和 `code`。
|
||||
|
||||
==================== Output ====================
|
||||
---------------------------------------
|
||||
{ printWidth: 80, proseWrap: "always" }
|
||||
---------------------------------------
|
||||
<!-- Whitespace around Chinese / Japanese: a line break between CJ characters is nothing,
|
||||
a space next to CJ text never breaks, Korean behaves like Latin -->
|
||||
<!-- Whitespace around Chinese / Japanese: a space next to CJ text never breaks, Korean behaves like Latin
|
||||
(a line break next to CJ text: DIVERGENCES.md#cj-line-break) -->
|
||||
|
||||
日本語の文章は改行しても空白になりません。
|
||||
|
||||
漢字と English の間に空白を置く文書では、改行も English の前で空白になります。
|
||||
|
||||
漢字とEnglishの間に空白を置かない文書では、改行はEnglishの前でも消えます。
|
||||
漢字と English の間に空白を置く文書でも、その空白では折り返さずに、段落の終わりまで一行のまま続きます。
|
||||
|
||||
한국어는 라틴 문자처럼 띄어쓰기로 단어를 나눕니다.
|
||||
|
||||
営業時間は月〜金の9時〜18時です。詳しくは……後日お知らせします……
|
||||
|
||||
括弧の前 (paren) と後ろ、ASCII 記号の :: 前後は空白になります。
|
||||
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does:
|
||||
the whitespace there may break and a line break stays a space
|
||||
(joined, `:www.` would no longer be an autolink literal) -->
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does: the whitespace there may break -->
|
||||
|
||||
使用方法请参考文档中的说明,如果需要了解更多信息,请参考
|
||||
[应用部署](./deployment.md)。
|
||||
|
||||
更多细节,请参考文档: www.example.com 和 `code`。
|
||||
|
||||
----------------------------------------
|
||||
{ printWidth: 100, proseWrap: "always" }
|
||||
----------------------------------------
|
||||
<!-- Whitespace around Chinese / Japanese: a line break between CJ characters is nothing,
|
||||
a space next to CJ text never breaks, Korean behaves like Latin -->
|
||||
<!-- Whitespace around Chinese / Japanese: a space next to CJ text never breaks, Korean behaves like Latin
|
||||
(a line break next to CJ text: DIVERGENCES.md#cj-line-break) -->
|
||||
|
||||
日本語の文章は改行しても空白になりません。
|
||||
|
||||
漢字と English の間に空白を置く文書では、改行も English の前で空白になります。
|
||||
|
||||
漢字とEnglishの間に空白を置かない文書では、改行はEnglishの前でも消えます。
|
||||
漢字と English の間に空白を置く文書でも、その空白では折り返さずに、段落の終わりまで一行のまま続きます。
|
||||
|
||||
한국어는 라틴 문자처럼 띄어쓰기로 단어를 나눕니다.
|
||||
|
||||
営業時間は月〜金の9時〜18時です。詳しくは……後日お知らせします……
|
||||
|
||||
括弧の前 (paren) と後ろ、ASCII 記号の :: 前後は空白になります。
|
||||
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does:
|
||||
the whitespace there may break and a line break stays a space
|
||||
(joined, `:www.` would no longer be an autolink literal) -->
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does: the whitespace there may break -->
|
||||
|
||||
使用方法请参考文档中的说明,如果需要了解更多信息,请参考 [应用部署](./deployment.md)。
|
||||
|
||||
更多细节,请参考文档: www.example.com 和 `code`。
|
||||
|
||||
--------------------------------------
|
||||
{ printWidth: 80, proseWrap: "never" }
|
||||
--------------------------------------
|
||||
<!-- Whitespace around Chinese / Japanese: a line break between CJ characters is nothing,
|
||||
a space next to CJ text never breaks, Korean behaves like Latin -->
|
||||
<!-- Whitespace around Chinese / Japanese: a space next to CJ text never breaks, Korean behaves like Latin
|
||||
(a line break next to CJ text: DIVERGENCES.md#cj-line-break) -->
|
||||
|
||||
日本語の文章は改行しても空白になりません。
|
||||
|
||||
漢字と English の間に空白を置く文書では、改行も English の前で空白になります。
|
||||
|
||||
漢字とEnglishの間に空白を置かない文書では、改行はEnglishの前でも消えます。
|
||||
漢字と English の間に空白を置く文書でも、その空白では折り返さずに、段落の終わりまで一行のまま続きます。
|
||||
|
||||
한국어는 라틴 문자처럼 띄어쓰기로 단어를 나눕니다.
|
||||
|
||||
営業時間は月〜金の9時〜18時です。詳しくは……後日お知らせします……
|
||||
|
||||
括弧の前 (paren) と後ろ、ASCII 記号の :: 前後は空白になります。
|
||||
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does:
|
||||
the whitespace there may break and a line break stays a space
|
||||
(joined, `:www.` would no longer be an autolink literal) -->
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does: the whitespace there may break -->
|
||||
|
||||
使用方法请参考文档中的说明,如果需要了解更多信息,请参考 [应用部署](./deployment.md)。
|
||||
|
||||
更多细节,请参考文档: www.example.com 和 `code`。
|
||||
|
||||
---------------------------------------
|
||||
{ printWidth: 100, proseWrap: "never" }
|
||||
---------------------------------------
|
||||
<!-- Whitespace around Chinese / Japanese: a line break between CJ characters is nothing,
|
||||
a space next to CJ text never breaks, Korean behaves like Latin -->
|
||||
<!-- Whitespace around Chinese / Japanese: a space next to CJ text never breaks, Korean behaves like Latin
|
||||
(a line break next to CJ text: DIVERGENCES.md#cj-line-break) -->
|
||||
|
||||
日本語の文章は改行しても空白になりません。
|
||||
|
||||
漢字と English の間に空白を置く文書では、改行も English の前で空白になります。
|
||||
|
||||
漢字とEnglishの間に空白を置かない文書では、改行はEnglishの前でも消えます。
|
||||
漢字と English の間に空白を置く文書でも、その空白では折り返さずに、段落の終わりまで一行のまま続きます。
|
||||
|
||||
한국어는 라틴 문자처럼 띄어쓰기로 단어를 나눕니다.
|
||||
|
||||
営業時間は月〜金の9時〜18時です。詳しくは……後日お知らせします……
|
||||
|
||||
括弧の前 (paren) と後ろ、ASCII 記号の :: 前後は空白になります。
|
||||
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does:
|
||||
the whitespace there may break and a line break stays a space
|
||||
(joined, `:www.` would no longer be an autolink literal) -->
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does: the whitespace there may break -->
|
||||
|
||||
使用方法请参考文档中的说明,如果需要了解更多信息,请参考 [应用部署](./deployment.md)。
|
||||
|
||||
更多细节,请参考文档: www.example.com 和 `code`。
|
||||
|
||||
-----------------------------------------
|
||||
{ printWidth: 80, proseWrap: "preserve" }
|
||||
-----------------------------------------
|
||||
<!-- Whitespace around Chinese / Japanese: a line break between CJ characters is nothing,
|
||||
a space next to CJ text never breaks, Korean behaves like Latin -->
|
||||
<!-- Whitespace around Chinese / Japanese: a space next to CJ text never breaks, Korean behaves like Latin
|
||||
(a line break next to CJ text: DIVERGENCES.md#cj-line-break) -->
|
||||
|
||||
日本語の文章は
|
||||
改行しても
|
||||
空白になりません。
|
||||
|
||||
漢字と English の間に空白を置く文書では、改行も
|
||||
English の前で空白になります。
|
||||
|
||||
漢字とEnglishの間に空白を置かない文書では、改行は
|
||||
Englishの前でも消えます。
|
||||
漢字と English の間に空白を置く文書でも、その空白では折り返さずに、段落の終わりまで一行のまま続きます。
|
||||
|
||||
한국어는 라틴 문자처럼
|
||||
띄어쓰기로 단어를 나눕니다.
|
||||
|
||||
営業時間は月〜
|
||||
金の9時〜
|
||||
18時です。詳しくは……
|
||||
後日お知らせします……
|
||||
|
||||
括弧の前 (paren) と後ろ、ASCII 記号の
|
||||
:: 前後は空白になります。
|
||||
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does:
|
||||
the whitespace there may break and a line break stays a space
|
||||
(joined, `:www.` would no longer be an autolink literal) -->
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does: the whitespace there may break -->
|
||||
|
||||
使用方法请参考文档中的说明,如果需要了解更多信息,请参考 [应用部署](./deployment.md)。
|
||||
|
||||
更多细节,请参考文档:
|
||||
www.example.com 和 `code`。
|
||||
|
||||
------------------------------------------
|
||||
{ printWidth: 100, proseWrap: "preserve" }
|
||||
------------------------------------------
|
||||
<!-- Whitespace around Chinese / Japanese: a line break between CJ characters is nothing,
|
||||
a space next to CJ text never breaks, Korean behaves like Latin -->
|
||||
<!-- Whitespace around Chinese / Japanese: a space next to CJ text never breaks, Korean behaves like Latin
|
||||
(a line break next to CJ text: DIVERGENCES.md#cj-line-break) -->
|
||||
|
||||
日本語の文章は
|
||||
改行しても
|
||||
空白になりません。
|
||||
|
||||
漢字と English の間に空白を置く文書では、改行も
|
||||
English の前で空白になります。
|
||||
|
||||
漢字とEnglishの間に空白を置かない文書では、改行は
|
||||
Englishの前でも消えます。
|
||||
漢字と English の間に空白を置く文書でも、その空白では折り返さずに、段落の終わりまで一行のまま続きます。
|
||||
|
||||
한국어는 라틴 문자처럼
|
||||
띄어쓰기로 단어를 나눕니다.
|
||||
|
||||
営業時間は月〜
|
||||
金の9時〜
|
||||
18時です。詳しくは……
|
||||
後日お知らせします……
|
||||
|
||||
括弧の前 (paren) と後ろ、ASCII 記号の
|
||||
:: 前後は空白になります。
|
||||
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does:
|
||||
the whitespace there may break and a line break stays a space
|
||||
(joined, `:www.` would no longer be an autolink literal) -->
|
||||
<!-- The rules stop at another node's edge, as Prettier's sentence does: the whitespace there may break -->
|
||||
|
||||
使用方法请参考文档中的说明,如果需要了解更多信息,请参考 [应用部署](./deployment.md)。
|
||||
|
||||
更多细节,请参考文档:
|
||||
www.example.com 和 `code`。
|
||||
|
||||
===================== End =====================
|
||||
@@ -2,7 +2,7 @@
|
||||
source: crates/oxc_formatter_markdown/tests/conformance.rs
|
||||
expression: report
|
||||
---
|
||||
markdown compatibility: 1263/1275 (99.06%), 0 files skipped
|
||||
markdown compatibility: 1262/1275 (98.98%), 0 files skipped
|
||||
|
||||
# Failed
|
||||
|
||||
@@ -19,4 +19,5 @@ markdown compatibility: 1263/1275 (99.06%), 0 files skipped
|
||||
| markdown/commonmark-test-suite/snippet: example-54.md (Thematic breaks) | 💥 | 0.00% |
|
||||
| markdown/link/encodedLink.md | 💥 | 86.67% |
|
||||
| markdown/paragraph/cjk.md | 💥💥✨ | 60.00% |
|
||||
| markdown/splitCjkText/symbolSpaceNewLine.md | 💥 | 20.25% |
|
||||
| markdown/thematicBreak/simple.md | 💥 | 0.00% |
|
||||
Reference in new issue
Block a user