feat(send): clip web URLs into self-contained EPUBs via Tauri webview (#4241)

Builds the URL-clipping path of the "Send to Readest" feature: paste a
link, the renderer ingests the rendered page, and a self-contained EPUB
lands in the library. No server proxy, no external CDN refs left in the
EPUB once it's saved.

Architecture

- New Rust `clip_url` command spawns a hidden Tauri WebviewWindow at the
  target URL with a real Chrome UA + WebKit fingerprint mask, so TLS-
  fingerprint and JS-challenge walls (Cloudflare, Medium, X, WeChat MP)
  resolve naturally instead of bouncing the server proxy.
- Capture transport is URL-payload navigation to a one-shot
  127.0.0.1:RANDOM_PORT/clip/{token}?d={url-safe-base64} listener.
  Top-level navigation isn't governed by CSP connect-src / form-action /
  WebKit Private Network Access — the four earlier transports
  (fetch, <form>, custom URI scheme, window.name) were each blocked by
  one of those.
- Page-to-EPUB bundler (`assetBundler`) walks <img>/<picture> with
  src → data-src → data-original → data-srcset → srcset fallback so lazy-
  loading sites don't ship a 60px LQIP; fetches assets in parallel with a
  per-asset timeout + per-asset/total caps; failed images degrade to alt-
  text placeholders. A per-site rules table (seeded with WeChat MP) + a
  selector fallback catches articles Readability misextracts. Builder
  prepends the article <h1> + byline so the EPUB has a proper opening.
- Nested EPUB TOC built from h1–h6.

UI surfaces

- "From Web URL" entry in the library Import menu, gated to Tauri; web
  build hides the URL field and points at the browser extension.
- `ImportFromUrlDialog` with auto-height (overrides Dialog's `sm:h-[65%]`
  default) and a dim placeholder for the URL field.
- Clip webview window styled to match Readest's main window — macOS
  decorations + overlay title bar; other desktops decorationless with a
  drop shadow; native background + in-page loading overlay pick up the
  caller's `themeCode.bg`/`fg` so light/dark/eink/custom themes all
  render correctly. Title localised, all five overlay/title strings
  translated across 33 locales.

Notes

- Gates the macOS traffic-light positioner to main/reader-* windows so
  the decorationless clip window no longer null-derefs in
  `position_traffic_lights`.
- Stricter validation across the path: schemes restricted to http/https,
  hex-color parsing rejects malformed values, server endpoint returns
  400 on missing/invalid base64.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Huang Xin
2026-05-21 01:48:09 +08:00
committed by GitHub
parent 3825f355a7
commit a1279a65ce
59 changed files with 2727 additions and 123 deletions
@@ -1621,5 +1621,17 @@
"Import all into library": "استيراد الكل إلى المكتبة",
"Recursively add every matching file directly to the library.": "إضافة كل ملف مطابق مباشرةً إلى المكتبة بشكل تكراري.",
"OK": "موافق",
"No matching books found in the selected folder.": "لم يتم العثور على كتب مطابقة في المجلد المحدد."
"No matching books found in the selected folder.": "لم يتم العثور على كتب مطابقة في المجلد المحدد.",
"Send a web article": "إرسال مقالة من الويب",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "ثبّت إضافة Readest للمتصفح لإرسال المقالة التي تقرأها إلى مكتبتك — تلتقط الصفحة من متصفحك لذا تعمل المواقع المدفوعة والتي تتطلب تسجيل الدخول أيضًا.",
"Enter a URL starting with http:// or https://": "أدخل رابطًا يبدأ بـ http:// أو https://",
"Import": "استيراد",
"Import from Web URL": "استيراد من رابط ويب",
"From Web URL": "من رابط الويب",
"Paste an article link. Readest clips the page and saves it to your library.": "الصق رابط مقالة. سيلتقط Readest الصفحة ويحفظها في مكتبتك.",
"Saving to your Readest library…": "جارٍ الحفظ في مكتبة Readest الخاصة بك…",
"Saving to Readest": "جارٍ الحفظ في Readest",
"Loading article…": "جارٍ تحميل المقالة…",
"Capturing article…": "جارٍ التقاط المقالة…",
"Saved to Readest": "تم الحفظ في Readest"
}
@@ -1509,5 +1509,17 @@
"Import all into library": "সব লাইব্রেরিতে আমদানি করুন",
"Recursively add every matching file directly to the library.": "প্রতিটি মিলিত ফাইলকে পুনরাবৃত্তিকভাবে সরাসরি লাইব্রেরিতে যোগ করুন।",
"OK": "ঠিক আছে",
"No matching books found in the selected folder.": "নির্বাচিত ফোল্ডারে কোনো মিলিত বই পাওয়া যায়নি।"
"No matching books found in the selected folder.": "নির্বাচিত ফোল্ডারে কোনো মিলিত বই পাওয়া যায়নি।",
"Send a web article": "একটি ওয়েব নিবন্ধ পাঠান",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "আপনি যে নিবন্ধটি পড়ছেন তা আপনার লাইব্রেরিতে পাঠাতে Readest ব্রাউজার এক্সটেনশন ইনস্টল করুন — এটি আপনার ব্রাউজার থেকে পৃষ্ঠাটি ক্লিপ করে, তাই পেওয়াল এবং শুধুমাত্র-লগইন সাইটগুলিও কাজ করে।",
"Enter a URL starting with http:// or https://": "http:// বা https:// দিয়ে শুরু হওয়া একটি URL লিখুন",
"Import": "আমদানি করুন",
"Import from Web URL": "ওয়েব URL থেকে আমদানি করুন",
"From Web URL": "ওয়েব URL থেকে",
"Paste an article link. Readest clips the page and saves it to your library.": "একটি নিবন্ধের লিঙ্ক পেস্ট করুন। Readest পৃষ্ঠাটি ক্যাপচার করে আপনার লাইব্রেরিতে সংরক্ষণ করবে।",
"Saving to your Readest library…": "আপনার Readest লাইব্রেরিতে সংরক্ষণ করা হচ্ছে…",
"Saving to Readest": "Readest-এ সংরক্ষণ করা হচ্ছে",
"Loading article…": "নিবন্ধ লোড হচ্ছে…",
"Capturing article…": "নিবন্ধ ক্যাপচার করা হচ্ছে…",
"Saved to Readest": "Readest-এ সংরক্ষিত"
}
@@ -1481,5 +1481,17 @@
"Import all into library": "ཡོངས་སུ་དཔེ་མཛོད་དུ་ནང་འདྲེན་བྱེད།",
"Recursively add every matching file directly to the library.": "མཐུན་པའི་ཡིག་ཆ་ཡོངས་སུ་དཔེ་མཛོད་དུ་ཐད་ཀར་ལོག་སྣོན་བྱས་ཏེ་ཁ་སྣོན་གྱིས།",
"OK": "ཡིན།",
"No matching books found in the selected folder.": "བདམས་པའི་ཡིག་སྣོད་ནང་མཐུན་པའི་དཔེ་ཆ་ག་ཡང་མ་རྙེད།"
"No matching books found in the selected folder.": "བདམས་པའི་ཡིག་སྣོད་ནང་མཐུན་པའི་དཔེ་ཆ་ག་ཡང་མ་རྙེད།",
"Send a web article": "དྲ་ངོས་ཀྱི་གཏམ་རྩོམ་སྐུར་སྤྲོད་པ",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "ཁྱེད་ཀྱི་ཀློག་བཞིན་པའི་གཏམ་རྩོམ་ཁྱེད་ཀྱི་དཔེ་མཛོད་དུ་སྐུར་འདོན་བྱེད་པར་Readest་ཕྲ་མེག་གི་ཁ་སྣོན་གཏོགས་གཞུང་སྒྲིག་འཇུག་གནང་རོགས — དེས་ཁྱེད་ཀྱི་ཁ་པར་ནང་ནས་ཤོག་ངོས་ལེན་པས་ངུལ་སྤྲོད་དགོས་པ་དང་ཐོ་འགོད་རྐྱང་གི་དྲ་ངོས་དག་ཀྱང་ལས་ཀ་བྱེད་ཐུབ།",
"Enter a URL starting with http:// or https://": "http:// ཡང་ན་ https:// ནས་འགོ་འཛུགས་པའི་URL་ཞིག་འཇུག",
"Import": "ནང་འདྲེན།",
"Import from Web URL": "དྲ་ངོས་ཀྱི་URL་ནས་ནང་འདྲེན།",
"From Web URL": "དྲ་ངོས་ཀྱི་URL་ནས།",
"Paste an article link. Readest clips the page and saves it to your library.": "རྩོམ་ཡིག་གི་འབྲེལ་མཐུད་ཕབ་སྦྱར་གནང༌། Readest གྱིས་ཤོག་ངོས་འཛིན་ནས་ཁྱེད་ཀྱི་དཔེ་མཛོད་དུ་ཉར་འཇོག་བྱེད།",
"Saving to your Readest library…": "ཁྱེད་ཀྱི་ Readest དཔེ་མཛོད་དུ་ཉར་བཞིན་པ་…",
"Saving to Readest": "Readest དུ་ཉར་བཞིན་པ་",
"Loading article…": "རྩོམ་ཡིག་སྦྱར་བཞིན་པ་…",
"Capturing article…": "རྩོམ་ཡིག་འཛིན་བཞིན་པ་…",
"Saved to Readest": "Readest དུ་ཉར་ཟིན།"
}
@@ -1509,5 +1509,17 @@
"Import all into library": "Alles in Bibliothek importieren",
"Recursively add every matching file directly to the library.": "Alle passenden Dateien rekursiv direkt zur Bibliothek hinzufügen.",
"OK": "OK",
"No matching books found in the selected folder.": "Keine passenden Bücher im ausgewählten Ordner gefunden."
"No matching books found in the selected folder.": "Keine passenden Bücher im ausgewählten Ordner gefunden.",
"Send a web article": "Webartikel senden",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Installiere die Readest-Browsererweiterung, um den Artikel, den du gerade liest, an deine Bibliothek zu senden — sie schneidet die Seite aus deinem Browser aus, sodass auch Bezahlschranken und Login-Sites funktionieren.",
"Enter a URL starting with http:// or https://": "Gib eine URL ein, die mit http:// oder https:// beginnt",
"Import": "Importieren",
"Import from Web URL": "Aus Webadresse importieren",
"From Web URL": "Von Webadresse",
"Paste an article link. Readest clips the page and saves it to your library.": "Fügen Sie einen Artikel-Link ein. Readest erfasst die Seite und speichert sie in Ihrer Bibliothek.",
"Saving to your Readest library…": "In Ihrer Readest-Bibliothek speichern…",
"Saving to Readest": "Wird in Readest gespeichert",
"Loading article…": "Artikel wird geladen…",
"Capturing article…": "Artikel wird erfasst…",
"Saved to Readest": "In Readest gespeichert"
}
@@ -1509,5 +1509,17 @@
"Import all into library": "Εισαγωγή όλων στη βιβλιοθήκη",
"Recursively add every matching file directly to the library.": "Αναδρομική προσθήκη κάθε αντίστοιχου αρχείου απευθείας στη βιβλιοθήκη.",
"OK": "OK",
"No matching books found in the selected folder.": "Δεν βρέθηκαν αντίστοιχα βιβλία στον επιλεγμένο φάκελο."
"No matching books found in the selected folder.": "Δεν βρέθηκαν αντίστοιχα βιβλία στον επιλεγμένο φάκελο.",
"Send a web article": "Αποστολή άρθρου από τον ιστό",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Εγκαταστήστε την επέκταση Readest για το πρόγραμμα περιήγησης για να στείλετε το άρθρο που διαβάζετε στη βιβλιοθήκη σας — αποκόπτει τη σελίδα από τον φυλλομετρητή σας ώστε να λειτουργούν και ιστότοποι με συνδρομή ή υποχρεωτική σύνδεση.",
"Enter a URL starting with http:// or https://": "Εισαγάγετε ένα URL που ξεκινά με http:// ή https://",
"Import": "Εισαγωγή",
"Import from Web URL": "Εισαγωγή από διεύθυνση ιστού",
"From Web URL": "Από διεύθυνση ιστού",
"Paste an article link. Readest clips the page and saves it to your library.": "Επικολλήστε έναν σύνδεσμο άρθρου. Το Readest αποτυπώνει τη σελίδα και την αποθηκεύει στη βιβλιοθήκη σας.",
"Saving to your Readest library…": "Αποθήκευση στη βιβλιοθήκη Readest…",
"Saving to Readest": "Αποθήκευση στο Readest",
"Loading article…": "Φόρτωση άρθρου…",
"Capturing article…": "Λήψη άρθρου…",
"Saved to Readest": "Αποθηκεύτηκε στο Readest"
}
@@ -1537,5 +1537,17 @@
"Import all into library": "Importar todo a la biblioteca",
"Recursively add every matching file directly to the library.": "Añadir recursivamente cada archivo coincidente directamente a la biblioteca.",
"OK": "OK",
"No matching books found in the selected folder.": "No se encontraron libros que coincidan en la carpeta seleccionada."
"No matching books found in the selected folder.": "No se encontraron libros que coincidan en la carpeta seleccionada.",
"Send a web article": "Enviar un artículo web",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Instala la extensión de navegador Readest para enviar el artículo que estás leyendo a tu biblioteca — recorta la página desde tu navegador, así que los sitios con muro de pago o que requieren inicio de sesión también funcionan.",
"Enter a URL starting with http:// or https://": "Introduce una URL que empiece por http:// o https://",
"Import": "Importar",
"Import from Web URL": "Importar desde URL web",
"From Web URL": "Desde URL web",
"Paste an article link. Readest clips the page and saves it to your library.": "Pega el enlace de un artículo. Readest captura la página y la guarda en tu biblioteca.",
"Saving to your Readest library…": "Guardando en tu biblioteca de Readest…",
"Saving to Readest": "Guardando en Readest",
"Loading article…": "Cargando artículo…",
"Capturing article…": "Capturando artículo…",
"Saved to Readest": "Guardado en Readest"
}
@@ -1509,5 +1509,17 @@
"Import all into library": "وارد کردن همه به کتابخانه",
"Recursively add every matching file directly to the library.": "افزودن بازگشتی همه فایل‌های مطابق به‌طور مستقیم به کتابخانه.",
"OK": "تأیید",
"No matching books found in the selected folder.": "هیچ کتاب مطابقی در پوشه انتخاب‌شده یافت نشد."
"No matching books found in the selected folder.": "هیچ کتاب مطابقی در پوشه انتخاب‌شده یافت نشد.",
"Send a web article": "ارسال یک مقاله وب",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "افزونه مرورگر Readest را نصب کنید تا مقاله‌ای را که می‌خوانید به کتابخانه خود ارسال کنید — صفحه را از مرورگر شما برش می‌دهد، بنابراین سایت‌های پولی و فقط ورود نیز کار می‌کنند.",
"Enter a URL starting with http:// or https://": "یک URL وارد کنید که با http:// یا https:// شروع شود",
"Import": "وارد کردن",
"Import from Web URL": "وارد کردن از آدرس وب",
"From Web URL": "از آدرس وب",
"Paste an article link. Readest clips the page and saves it to your library.": "پیوند یک مقاله را الصاق کنید. Readest صفحه را گرفته و در کتابخانه شما ذخیره می‌کند.",
"Saving to your Readest library…": "در حال ذخیره در کتابخانه Readest شما…",
"Saving to Readest": "در حال ذخیره در Readest",
"Loading article…": "در حال بارگذاری مقاله…",
"Capturing article…": "در حال دریافت مقاله…",
"Saved to Readest": "در Readest ذخیره شد"
}
@@ -1537,5 +1537,17 @@
"Import all into library": "Tout importer dans la bibliothèque",
"Recursively add every matching file directly to the library.": "Ajouter récursivement tous les fichiers correspondants directement à la bibliothèque.",
"OK": "OK",
"No matching books found in the selected folder.": "Aucun livre correspondant trouvé dans le dossier sélectionné."
"No matching books found in the selected folder.": "Aucun livre correspondant trouvé dans le dossier sélectionné.",
"Send a web article": "Envoyer un article web",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Installez lextension de navigateur Readest pour envoyer larticle que vous lisez à votre bibliothèque — elle découpe la page depuis votre navigateur, donc les sites payants et nécessitant une connexion fonctionnent aussi.",
"Enter a URL starting with http:// or https://": "Saisissez une URL commençant par http:// ou https://",
"Import": "Importer",
"Import from Web URL": "Importer depuis une URL web",
"From Web URL": "Depuis une URL web",
"Paste an article link. Readest clips the page and saves it to your library.": "Collez le lien d'un article. Readest capture la page et l'enregistre dans votre bibliothèque.",
"Saving to your Readest library…": "Enregistrement dans votre bibliothèque Readest…",
"Saving to Readest": "Enregistrement dans Readest",
"Loading article…": "Chargement de l'article…",
"Capturing article…": "Capture de l'article…",
"Saved to Readest": "Enregistré dans Readest"
}
@@ -1537,5 +1537,17 @@
"Import all into library": "ייבא הכול לספרייה",
"Recursively add every matching file directly to the library.": "הוסף באופן רקורסיבי כל קובץ תואם ישירות לספרייה.",
"OK": "אישור",
"No matching books found in the selected folder.": "לא נמצאו ספרים תואמים בתיקייה שנבחרה."
"No matching books found in the selected folder.": "לא נמצאו ספרים תואמים בתיקייה שנבחרה.",
"Send a web article": "שלח מאמר אינטרנט",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "התקן את הרחבת הדפדפן של Readest כדי לשלוח את המאמר שאתה קורא לספרייה שלך — היא חותכת את הדף מהדפדפן שלך כך שגם אתרים עם חומת תשלום ואתרים שדורשים התחברות עובדים.",
"Enter a URL starting with http:// or https://": "הזן URL שמתחיל ב-http:// או ב-https://",
"Import": "יבוא",
"Import from Web URL": "יבוא מ-URL של אתר",
"From Web URL": "מ-URL של אתר",
"Paste an article link. Readest clips the page and saves it to your library.": "הדבק קישור למאמר. Readest יצלם את הדף וישמור אותו בספרייה שלך.",
"Saving to your Readest library…": "שומר בספריית Readest שלך…",
"Saving to Readest": "שומר ב-Readest",
"Loading article…": "טוען מאמר…",
"Capturing article…": "לוכד מאמר…",
"Saved to Readest": "נשמר ב-Readest"
}
@@ -1509,5 +1509,17 @@
"Import all into library": "सभी को लाइब्रेरी में आयात करें",
"Recursively add every matching file directly to the library.": "हर मेल खाने वाली फ़ाइल को सीधे लाइब्रेरी में पुनरावर्ती रूप से जोड़ें।",
"OK": "ठीक है",
"No matching books found in the selected folder.": "चयनित फ़ोल्डर में कोई मेल खाती पुस्तक नहीं मिली।"
"No matching books found in the selected folder.": "चयनित फ़ोल्डर में कोई मेल खाती पुस्तक नहीं मिली।",
"Send a web article": "एक वेब लेख भेजें",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "जो लेख आप पढ़ रहे हैं उसे अपनी लाइब्रेरी में भेजने के लिए Readest ब्राउज़र एक्सटेंशन इंस्टॉल करें — यह आपके ब्राउज़र से पृष्ठ क्लिप करता है, इसलिए पेवॉल और लॉगिन-केवल साइटें भी काम करती हैं।",
"Enter a URL starting with http:// or https://": "http:// या https:// से शुरू होने वाला URL दर्ज करें",
"Import": "आयात करें",
"Import from Web URL": "वेब URL से आयात करें",
"From Web URL": "वेब URL से",
"Paste an article link. Readest clips the page and saves it to your library.": "एक लेख का लिंक चिपकाएं। Readest पेज कैप्चर करके आपकी लाइब्रेरी में सहेज देगा।",
"Saving to your Readest library…": "आपकी Readest लाइब्रेरी में सहेजा जा रहा है…",
"Saving to Readest": "Readest में सहेजा जा रहा है",
"Loading article…": "लेख लोड हो रहा है…",
"Capturing article…": "लेख कैप्चर हो रहा है…",
"Saved to Readest": "Readest में सहेजा गया"
}
@@ -1509,5 +1509,17 @@
"Import all into library": "Mindent importálni a könyvtárba",
"Recursively add every matching file directly to the library.": "Minden megfelelő fájl rekurzív hozzáadása közvetlenül a könyvtárhoz.",
"OK": "OK",
"No matching books found in the selected folder.": "Nem található megfelelő könyv a kiválasztott mappában."
"No matching books found in the selected folder.": "Nem található megfelelő könyv a kiválasztott mappában.",
"Send a web article": "Webcikk küldése",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Telepítsd a Readest böngészőbővítményt, hogy a jelenleg olvasott cikket a könyvtáradba küldhesd — kivágja az oldalt a böngésződből, így a fizetős és csak bejelentkezéssel elérhető oldalak is működnek.",
"Enter a URL starting with http:// or https://": "Adj meg egy URL-t, ami http://-vel vagy https://-vel kezdődik",
"Import": "Importálás",
"Import from Web URL": "Importálás webcímről",
"From Web URL": "Webcímről",
"Paste an article link. Readest clips the page and saves it to your library.": "Illesszen be egy cikkre mutató hivatkozást. A Readest rögzíti az oldalt, és elmenti a könyvtárába.",
"Saving to your Readest library…": "Mentés a Readest könyvtárba…",
"Saving to Readest": "Mentés a Readestbe",
"Loading article…": "Cikk betöltése…",
"Capturing article…": "Cikk rögzítése…",
"Saved to Readest": "Mentve a Readestbe"
}
@@ -1481,5 +1481,17 @@
"Import all into library": "Impor semua ke pustaka",
"Recursively add every matching file directly to the library.": "Tambahkan secara rekursif setiap berkas yang cocok langsung ke pustaka.",
"OK": "OK",
"No matching books found in the selected folder.": "Tidak ada buku yang cocok ditemukan di folder yang dipilih."
"No matching books found in the selected folder.": "Tidak ada buku yang cocok ditemukan di folder yang dipilih.",
"Send a web article": "Kirim artikel web",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Pasang ekstensi peramban Readest untuk mengirim artikel yang Anda baca ke perpustakaan — ekstensi memotong halaman dari peramban Anda sehingga situs berbayar dan hanya-login pun berfungsi.",
"Enter a URL starting with http:// or https://": "Masukkan URL yang diawali dengan http:// atau https://",
"Import": "Impor",
"Import from Web URL": "Impor dari URL web",
"From Web URL": "Dari URL web",
"Paste an article link. Readest clips the page and saves it to your library.": "Tempel tautan artikel. Readest akan menangkap halaman dan menyimpannya ke pustaka Anda.",
"Saving to your Readest library…": "Menyimpan ke perpustakaan Readest Anda…",
"Saving to Readest": "Menyimpan ke Readest",
"Loading article…": "Memuat artikel…",
"Capturing article…": "Menangkap artikel…",
"Saved to Readest": "Tersimpan di Readest"
}
@@ -1537,5 +1537,17 @@
"Import all into library": "Importa tutto nella libreria",
"Recursively add every matching file directly to the library.": "Aggiungi ricorsivamente ogni file corrispondente direttamente alla libreria.",
"OK": "OK",
"No matching books found in the selected folder.": "Nessun libro corrispondente trovato nella cartella selezionata."
"No matching books found in the selected folder.": "Nessun libro corrispondente trovato nella cartella selezionata.",
"Send a web article": "Invia un articolo web",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Installa lestensione del browser Readest per inviare larticolo che stai leggendo alla tua biblioteca — ritaglia la pagina dal browser, così funzionano anche i siti con paywall o che richiedono il login.",
"Enter a URL starting with http:// or https://": "Inserisci un URL che inizia con http:// o https://",
"Import": "Importa",
"Import from Web URL": "Importa da URL web",
"From Web URL": "Da URL web",
"Paste an article link. Readest clips the page and saves it to your library.": "Incolla il link di un articolo. Readest acquisisce la pagina e la salva nella tua libreria.",
"Saving to your Readest library…": "Salvataggio nella tua libreria Readest…",
"Saving to Readest": "Salvataggio in Readest",
"Loading article…": "Caricamento dell'articolo…",
"Capturing article…": "Acquisizione dell'articolo…",
"Saved to Readest": "Salvato in Readest"
}
@@ -1481,5 +1481,17 @@
"Import all into library": "すべてライブラリに取り込む",
"Recursively add every matching file directly to the library.": "一致するすべてのファイルを再帰的にライブラリに直接追加します。",
"OK": "OK",
"No matching books found in the selected folder.": "選択したフォルダ内に一致する書籍が見つかりませんでした。"
"No matching books found in the selected folder.": "選択したフォルダ内に一致する書籍が見つかりませんでした。",
"Send a web article": "ウェブ記事を送信",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "読んでいる記事をライブラリに送るには、Readestのブラウザ拡張機能をインストールしてください — ブラウザからページをクリップするため、ペイウォールやログインが必要なサイトでも動作します。",
"Enter a URL starting with http:// or https://": "http:// または https:// で始まる URL を入力してください",
"Import": "インポート",
"Import from Web URL": "ウェブURLからインポート",
"From Web URL": "ウェブURLから",
"Paste an article link. Readest clips the page and saves it to your library.": "記事のリンクを貼り付けてください。Readest がページを取得してライブラリに保存します。",
"Saving to your Readest library…": "Readest ライブラリに保存中…",
"Saving to Readest": "Readest に保存中",
"Loading article…": "記事を読み込み中…",
"Capturing article…": "記事を取得中…",
"Saved to Readest": "Readest に保存しました"
}
@@ -1481,5 +1481,17 @@
"Import all into library": "모두 라이브러리로 가져오기",
"Recursively add every matching file directly to the library.": "일치하는 모든 파일을 재귀적으로 라이브러리에 직접 추가합니다.",
"OK": "확인",
"No matching books found in the selected folder.": "선택한 폴더에서 일치하는 책을 찾을 수 없습니다."
"No matching books found in the selected folder.": "선택한 폴더에서 일치하는 책을 찾을 수 없습니다.",
"Send a web article": "웹 기사 보내기",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "읽고 있는 기사를 라이브러리로 보내려면 Readest 브라우저 확장 프로그램을 설치하세요 — 브라우저에서 페이지를 잘라내므로 유료 결제벽이나 로그인이 필요한 사이트도 작동합니다.",
"Enter a URL starting with http:// or https://": "http:// 또는 https://로 시작하는 URL을 입력하세요",
"Import": "가져오기",
"Import from Web URL": "웹 URL에서 가져오기",
"From Web URL": "웹 URL에서",
"Paste an article link. Readest clips the page and saves it to your library.": "기사 링크를 붙여넣으세요. Readest가 페이지를 캡처하여 라이브러리에 저장합니다.",
"Saving to your Readest library…": "Readest 라이브러리에 저장 중…",
"Saving to Readest": "Readest에 저장 중",
"Loading article…": "기사 로드 중…",
"Capturing article…": "기사 캡처 중…",
"Saved to Readest": "Readest에 저장됨"
}
@@ -1481,5 +1481,17 @@
"Import all into library": "Import semua ke pustaka",
"Recursively add every matching file directly to the library.": "Tambah secara rekursif setiap fail yang sepadan terus ke pustaka.",
"OK": "OK",
"No matching books found in the selected folder.": "Tiada buku sepadan dijumpai dalam folder yang dipilih."
"No matching books found in the selected folder.": "Tiada buku sepadan dijumpai dalam folder yang dipilih.",
"Send a web article": "Hantar artikel web",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Pasang sambungan pelayar Readest untuk menghantar artikel yang anda baca ke perpustakaan anda — ia memotong halaman daripada pelayar anda jadi laman berbayar dan log-masuk-sahaja pun berfungsi.",
"Enter a URL starting with http:// or https://": "Masukkan URL yang bermula dengan http:// atau https://",
"Import": "Import",
"Import from Web URL": "Import dari URL web",
"From Web URL": "Dari URL web",
"Paste an article link. Readest clips the page and saves it to your library.": "Tampal pautan artikel. Readest akan menangkap halaman dan menyimpannya ke pustaka anda.",
"Saving to your Readest library…": "Menyimpan ke pustaka Readest anda…",
"Saving to Readest": "Menyimpan ke Readest",
"Loading article…": "Memuatkan artikel…",
"Capturing article…": "Menangkap artikel…",
"Saved to Readest": "Disimpan ke Readest"
}
@@ -1509,5 +1509,17 @@
"Import all into library": "Alles in bibliotheek importeren",
"Recursively add every matching file directly to the library.": "Voeg recursief elk overeenkomend bestand rechtstreeks toe aan de bibliotheek.",
"OK": "OK",
"No matching books found in the selected folder.": "Geen overeenkomende boeken gevonden in de geselecteerde map."
"No matching books found in the selected folder.": "Geen overeenkomende boeken gevonden in de geselecteerde map.",
"Send a web article": "Webartikel verzenden",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Installeer de Readest-browserextensie om het artikel dat je leest naar je bibliotheek te sturen — het knipt de pagina vanuit je browser, dus betaalde en alleen-met-inloggen sites werken ook.",
"Enter a URL starting with http:// or https://": "Voer een URL in die met http:// of https:// begint",
"Import": "Importeren",
"Import from Web URL": "Importeren vanaf web-URL",
"From Web URL": "Van web-URL",
"Paste an article link. Readest clips the page and saves it to your library.": "Plak een artikellink. Readest legt de pagina vast en slaat hem op in je bibliotheek.",
"Saving to your Readest library…": "Opslaan in je Readest-bibliotheek…",
"Saving to Readest": "Opslaan in Readest",
"Loading article…": "Artikel laden…",
"Capturing article…": "Artikel vastleggen…",
"Saved to Readest": "Opgeslagen in Readest"
}
@@ -1565,5 +1565,17 @@
"Import all into library": "Importuj wszystko do biblioteki",
"Recursively add every matching file directly to the library.": "Dodaj rekurencyjnie każdy pasujący plik bezpośrednio do biblioteki.",
"OK": "OK",
"No matching books found in the selected folder.": "Nie znaleziono pasujących książek w wybranym folderze."
"No matching books found in the selected folder.": "Nie znaleziono pasujących książek w wybranym folderze.",
"Send a web article": "Wyślij artykuł z internetu",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Zainstaluj rozszerzenie przeglądarki Readest, aby wysłać czytany artykuł do swojej biblioteki — wycina ono stronę z Twojej przeglądarki, więc działa też dla paywalli i stron wymagających logowania.",
"Enter a URL starting with http:// or https://": "Wprowadź URL zaczynający się od http:// lub https://",
"Import": "Importuj",
"Import from Web URL": "Importuj z adresu URL",
"From Web URL": "Z adresu URL",
"Paste an article link. Readest clips the page and saves it to your library.": "Wklej link do artykułu. Readest przechwyci stronę i zapisze ją w Twojej bibliotece.",
"Saving to your Readest library…": "Zapisywanie do biblioteki Readest…",
"Saving to Readest": "Zapisywanie w Readest",
"Loading article…": "Ładowanie artykułu…",
"Capturing article…": "Pobieranie artykułu…",
"Saved to Readest": "Zapisano w Readest"
}
@@ -1537,5 +1537,17 @@
"Import all into library": "Importar tudo para a biblioteca",
"Recursively add every matching file directly to the library.": "Adicionar recursivamente todo arquivo correspondente diretamente à biblioteca.",
"OK": "OK",
"No matching books found in the selected folder.": "Nenhum livro correspondente encontrado na pasta selecionada."
"No matching books found in the selected folder.": "Nenhum livro correspondente encontrado na pasta selecionada.",
"Send a web article": "Enviar um artigo da web",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Instale a extensão de navegador do Readest para enviar o artigo que você está lendo à sua biblioteca — ela recorta a página direto do seu navegador, então sites com paywall ou que exigem login também funcionam.",
"Enter a URL starting with http:// or https://": "Insira uma URL que comece com http:// ou https://",
"Import": "Importar",
"Import from Web URL": "Importar de URL da web",
"From Web URL": "De URL da web",
"Paste an article link. Readest clips the page and saves it to your library.": "Cole o link de um artigo. O Readest captura a página e a salva na sua biblioteca.",
"Saving to your Readest library…": "Salvando na sua biblioteca Readest…",
"Saving to Readest": "Salvando no Readest",
"Loading article…": "Carregando artigo…",
"Capturing article…": "Capturando artigo…",
"Saved to Readest": "Salvo no Readest"
}
@@ -1537,5 +1537,17 @@
"Import all into library": "Importar tudo para a biblioteca",
"Recursively add every matching file directly to the library.": "Adicionar recursivamente cada ficheiro correspondente diretamente à biblioteca.",
"OK": "OK",
"No matching books found in the selected folder.": "Não foram encontrados livros correspondentes na pasta selecionada."
"No matching books found in the selected folder.": "Não foram encontrados livros correspondentes na pasta selecionada.",
"Send a web article": "Enviar um artigo da web",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Instala a extensão de navegador do Readest para enviares o artigo que estás a ler para a tua biblioteca — recorta a página a partir do teu navegador, por isso sites pagos e só com login também funcionam.",
"Enter a URL starting with http:// or https://": "Introduz um URL que comece com http:// ou https://",
"Import": "Importar",
"Import from Web URL": "Importar de URL da web",
"From Web URL": "De URL da web",
"Paste an article link. Readest clips the page and saves it to your library.": "Cole o link de um artigo. O Readest captura a página e guarda-a na sua biblioteca.",
"Saving to your Readest library…": "A guardar na sua biblioteca Readest…",
"Saving to Readest": "A guardar no Readest",
"Loading article…": "A carregar o artigo…",
"Capturing article…": "A capturar o artigo…",
"Saved to Readest": "Guardado no Readest"
}
@@ -1537,5 +1537,17 @@
"Import all into library": "Importă tot în bibliotecă",
"Recursively add every matching file directly to the library.": "Adaugă recursiv fiecare fișier potrivit direct în bibliotecă.",
"OK": "OK",
"No matching books found in the selected folder.": "Nu s-au găsit cărți potrivite în folderul selectat."
"No matching books found in the selected folder.": "Nu s-au găsit cărți potrivite în folderul selectat.",
"Send a web article": "Trimite un articol web",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Instalează extensia de browser Readest pentru a trimite articolul pe care îl citești în biblioteca ta — taie pagina din browserul tău, așa că funcționează și pe site-urile cu plată sau care necesită autentificare.",
"Enter a URL starting with http:// or https://": "Introdu o adresă URL care începe cu http:// sau https://",
"Import": "Importă",
"Import from Web URL": "Importă dintr-un URL web",
"From Web URL": "Dintr-un URL web",
"Paste an article link. Readest clips the page and saves it to your library.": "Lipiți un link către articol. Readest captează pagina și o salvează în biblioteca dvs.",
"Saving to your Readest library…": "Se salvează în biblioteca dvs. Readest…",
"Saving to Readest": "Se salvează în Readest",
"Loading article…": "Se încarcă articolul…",
"Capturing article…": "Se capturează articolul…",
"Saved to Readest": "Salvat în Readest"
}
@@ -1565,5 +1565,17 @@
"Import all into library": "Импортировать всё в библиотеку",
"Recursively add every matching file directly to the library.": "Рекурсивно добавить каждый подходящий файл напрямую в библиотеку.",
"OK": "OK",
"No matching books found in the selected folder.": "В выбранной папке не найдено подходящих книг."
"No matching books found in the selected folder.": "В выбранной папке не найдено подходящих книг.",
"Send a web article": "Отправить веб-статью",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Установите расширение Readest для браузера, чтобы отправить читаемую статью в библиотеку — оно вырезает страницу из вашего браузера, поэтому платные и требующие входа сайты тоже работают.",
"Enter a URL starting with http:// or https://": "Введите URL, начинающийся с http:// или https://",
"Import": "Импорт",
"Import from Web URL": "Импорт по веб-адресу",
"From Web URL": "По веб-адресу",
"Paste an article link. Readest clips the page and saves it to your library.": "Вставьте ссылку на статью. Readest захватит страницу и сохранит её в вашей библиотеке.",
"Saving to your Readest library…": "Сохранение в вашу библиотеку Readest…",
"Saving to Readest": "Сохранение в Readest",
"Loading article…": "Загрузка статьи…",
"Capturing article…": "Захват статьи…",
"Saved to Readest": "Сохранено в Readest"
}
@@ -1509,5 +1509,17 @@
"Import all into library": "සියල්ල පුස්තකාලයට ආයාත කරන්න",
"Recursively add every matching file directly to the library.": "ගැලපෙන සෑම ගොනුවක්ම පුස්තකාලයට කෙළින්ම පුනරාවර්තනීයව එක් කරන්න.",
"OK": "හරි",
"No matching books found in the selected folder.": "තෝරාගත් ෆෝල්ඩරයේ ගැලපෙන පොත් කිසිවක් හමු නොවීය."
"No matching books found in the selected folder.": "තෝරාගත් ෆෝල්ඩරයේ ගැලපෙන පොත් කිසිවක් හමු නොවීය.",
"Send a web article": "වෙබ් ලිපියක් යවන්න",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "ඔබ කියවන ලිපිය ඔබේ පුස්තකාලයට යැවීමට Readest බ්‍රවුසර දිගුව ස්ථාපනය කරන්න — එය ඔබේ බ්‍රවුසරයෙන් පිටුව කපා ගන්නා නිසා ගෙවීම් අවශ්‍ය හා ඇතුළු වීම පමණක් ඇති වෙබ් අඩවි ද ක්‍රියා කරයි.",
"Enter a URL starting with http:// or https://": "http:// හෝ https:// වලින් ආරම්භ වන URL එකක් ඇතුළත් කරන්න",
"Import": "ආයාත කරන්න",
"Import from Web URL": "වෙබ් URL වෙතින් ආයාත කරන්න",
"From Web URL": "වෙබ් URL වෙතින්",
"Paste an article link. Readest clips the page and saves it to your library.": "ලිපියක සබැඳියක් අලවන්න. Readest පිටුව ග්‍රහණය කර ඔබගේ පුස්තකාලයට සුරකියි.",
"Saving to your Readest library…": "ඔබගේ Readest පුස්තකාලයට සුරකියි…",
"Saving to Readest": "Readest වෙත සුරකියි",
"Loading article…": "ලිපිය පූරණය වෙමින්…",
"Capturing article…": "ලිපිය ලබා ගනියි…",
"Saved to Readest": "Readest හි සුරැකිණි"
}
@@ -1565,5 +1565,17 @@
"Import all into library": "Uvozi vse v knjižnico",
"Recursively add every matching file directly to the library.": "Rekurzivno dodaj vsako ujemajočo datoteko neposredno v knjižnico.",
"OK": "V redu",
"No matching books found in the selected folder.": "V izbrani mapi ni najdenih ujemajočih se knjig."
"No matching books found in the selected folder.": "V izbrani mapi ni najdenih ujemajočih se knjig.",
"Send a web article": "Pošlji spletni članek",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Namesti razširitev brskalnika Readest, da pošlješ članek, ki ga bereš, v svojo knjižnico — izreže stran iz brskalnika, zato delujejo tudi plačljive in samo prijavljene strani.",
"Enter a URL starting with http:// or https://": "Vnesi URL, ki se začne s http:// ali https://",
"Import": "Uvozi",
"Import from Web URL": "Uvozi s spletnega URL",
"From Web URL": "S spletnega URL",
"Paste an article link. Readest clips the page and saves it to your library.": "Prilepite povezavo do članka. Readest zajame stran in jo shrani v vašo knjižnico.",
"Saving to your Readest library…": "Shranjevanje v vašo knjižnico Readest…",
"Saving to Readest": "Shranjevanje v Readest",
"Loading article…": "Nalaganje članka…",
"Capturing article…": "Zajemanje članka…",
"Saved to Readest": "Shranjeno v Readest"
}
@@ -1509,5 +1509,17 @@
"Import all into library": "Importera allt till biblioteket",
"Recursively add every matching file directly to the library.": "Lägg rekursivt till varje matchande fil direkt i biblioteket.",
"OK": "OK",
"No matching books found in the selected folder.": "Inga matchande böcker hittades i den valda mappen."
"No matching books found in the selected folder.": "Inga matchande böcker hittades i den valda mappen.",
"Send a web article": "Skicka en webbartikel",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Installera Readests webbläsartillägg för att skicka artikeln du läser till ditt bibliotek — det klipper sidan från din webbläsare så att betalda och inloggade sajter också fungerar.",
"Enter a URL starting with http:// or https://": "Ange en URL som börjar med http:// eller https://",
"Import": "Importera",
"Import from Web URL": "Importera från webbadress",
"From Web URL": "Från webbadress",
"Paste an article link. Readest clips the page and saves it to your library.": "Klistra in en artikellänk. Readest fångar sidan och sparar den i ditt bibliotek.",
"Saving to your Readest library…": "Sparar till ditt Readest-bibliotek…",
"Saving to Readest": "Sparar till Readest",
"Loading article…": "Laddar artikel…",
"Capturing article…": "Fångar artikel…",
"Saved to Readest": "Sparad till Readest"
}
@@ -1509,5 +1509,17 @@
"Import all into library": "அனைத்தையும் நூலகத்திற்கு இறக்குமதி செய்",
"Recursively add every matching file directly to the library.": "பொருந்தும் ஒவ்வொரு கோப்பையும் நேரடியாக நூலகத்திற்கு மீள்கூற்று முறையில் சேர்க்கவும்.",
"OK": "சரி",
"No matching books found in the selected folder.": "தேர்ந்தெடுக்கப்பட்ட கோப்புறையில் பொருந்தும் புத்தகங்கள் எதுவும் கிடைக்கவில்லை."
"No matching books found in the selected folder.": "தேர்ந்தெடுக்கப்பட்ட கோப்புறையில் பொருந்தும் புத்தகங்கள் எதுவும் கிடைக்கவில்லை.",
"Send a web article": "ஒரு வலை கட்டுரையை அனுப்பவும்",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "நீங்கள் வாசிக்கும் கட்டுரையை உங்கள் நூலகத்திற்கு அனுப்ப Readest உலாவி நீட்டிப்பை நிறுவவும் — இது உங்கள் உலாவியில் இருந்தே பக்கத்தை வெட்டுகிறது, எனவே கட்டண சுவர்கள் மற்றும் உள்நுழைவு மட்டுமே அனுமதிக்கும் தளங்களும் வேலை செய்யும்.",
"Enter a URL starting with http:// or https://": "http:// அல்லது https:// உடன் தொடங்கும் URL-ஐ உள்ளிடவும்",
"Import": "இறக்கு",
"Import from Web URL": "வலை URL-இலிருந்து இறக்குமதி செய்",
"From Web URL": "வலை URL-இலிருந்து",
"Paste an article link. Readest clips the page and saves it to your library.": "கட்டுரை இணைப்பை ஒட்டவும். Readest பக்கத்தை எடுத்து உங்கள் நூலகத்தில் சேமிக்கும்.",
"Saving to your Readest library…": "உங்கள் Readest நூலகத்தில் சேமிக்கப்படுகிறது…",
"Saving to Readest": "Readest-இல் சேமிக்கப்படுகிறது",
"Loading article…": "கட்டுரை ஏற்றப்படுகிறது…",
"Capturing article…": "கட்டுரை பெறப்படுகிறது…",
"Saved to Readest": "Readest-இல் சேமிக்கப்பட்டது"
}
@@ -1481,5 +1481,17 @@
"Import all into library": "นำเข้าทั้งหมดไปยังห้องสมุด",
"Recursively add every matching file directly to the library.": "เพิ่มทุกไฟล์ที่ตรงกันโดยตรงไปยังห้องสมุดแบบเรียกซ้ำ",
"OK": "ตกลง",
"No matching books found in the selected folder.": "ไม่พบหนังสือที่ตรงกันในโฟลเดอร์ที่เลือก"
"No matching books found in the selected folder.": "ไม่พบหนังสือที่ตรงกันในโฟลเดอร์ที่เลือก",
"Send a web article": "ส่งบทความเว็บ",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "ติดตั้งส่วนขยายเบราว์เซอร์ Readest เพื่อส่งบทความที่คุณกำลังอ่านไปยังคลังของคุณ — มันตัดหน้าจากเบราว์เซอร์ของคุณ ดังนั้นเว็บไซต์ที่มีกำแพงค่าใช้จ่ายและเฉพาะผู้ที่ล็อกอินก็ยังคงทำงาน",
"Enter a URL starting with http:// or https://": "ใส่ URL ที่ขึ้นต้นด้วย http:// หรือ https://",
"Import": "นำเข้า",
"Import from Web URL": "นำเข้าจาก URL เว็บ",
"From Web URL": "จาก URL เว็บ",
"Paste an article link. Readest clips the page and saves it to your library.": "วางลิงก์บทความ Readest จะจับภาพหน้าเว็บและบันทึกลงในไลบรารีของคุณ",
"Saving to your Readest library…": "กำลังบันทึกในไลบรารี Readest ของคุณ…",
"Saving to Readest": "กำลังบันทึกลงใน Readest",
"Loading article…": "กำลังโหลดบทความ…",
"Capturing article…": "กำลังจับภาพบทความ…",
"Saved to Readest": "บันทึกลงใน Readest แล้ว"
}
@@ -1509,5 +1509,17 @@
"Import all into library": "Tümünü kitaplığa aktar",
"Recursively add every matching file directly to the library.": "Eşleşen her dosyayı doğrudan kitaplığa özyinelemeli olarak ekle.",
"OK": "Tamam",
"No matching books found in the selected folder.": "Seçilen klasörde eşleşen kitap bulunamadı."
"No matching books found in the selected folder.": "Seçilen klasörde eşleşen kitap bulunamadı.",
"Send a web article": "Bir web makalesi gönder",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Okuduğunuz makaleyi kitaplığınıza göndermek için Readest tarayıcı eklentisini yükleyin — sayfayı tarayıcınızdan kırpar, böylece ödeme duvarlı ve yalnızca giriş yapılan siteler de çalışır.",
"Enter a URL starting with http:// or https://": "http:// veya https:// ile başlayan bir URL girin",
"Import": "İçe aktar",
"Import from Web URL": "Web URL'sinden içe aktar",
"From Web URL": "Web URL'sinden",
"Paste an article link. Readest clips the page and saves it to your library.": "Bir makale bağlantısı yapıştırın. Readest sayfayı yakalayıp kütüphanenize kaydeder.",
"Saving to your Readest library…": "Readest kütüphanenize kaydediliyor…",
"Saving to Readest": "Readest'e kaydediliyor",
"Loading article…": "Makale yükleniyor…",
"Capturing article…": "Makale yakalanıyor…",
"Saved to Readest": "Readest'e kaydedildi"
}
@@ -1565,5 +1565,17 @@
"Import all into library": "Імпортувати все до бібліотеки",
"Recursively add every matching file directly to the library.": "Рекурсивно додати кожен відповідний файл безпосередньо до бібліотеки.",
"OK": "OK",
"No matching books found in the selected folder.": "Не знайдено відповідних книг у вибраній папці."
"No matching books found in the selected folder.": "Не знайдено відповідних книг у вибраній папці.",
"Send a web article": "Надіслати веб-статтю",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Встановіть розширення браузера Readest, щоб надіслати статтю, яку ви читаєте, до вашої бібліотеки — воно вирізає сторінку з вашого браузера, тож працюють також платні та лише авторизовані сайти.",
"Enter a URL starting with http:// or https://": "Введіть URL, що починається з http:// або https://",
"Import": "Імпорт",
"Import from Web URL": "Імпорт з веб-адреси",
"From Web URL": "З веб-адреси",
"Paste an article link. Readest clips the page and saves it to your library.": "Вставте посилання на статтю. Readest захоплює сторінку та зберігає її у вашу бібліотеку.",
"Saving to your Readest library…": "Збереження у вашу бібліотеку Readest…",
"Saving to Readest": "Збереження в Readest",
"Loading article…": "Завантаження статті…",
"Capturing article…": "Захоплення статті…",
"Saved to Readest": "Збережено в Readest"
}
@@ -1509,5 +1509,17 @@
"Import all into library": "Hammasini kutubxonaga import qiling",
"Recursively add every matching file directly to the library.": "Har bir mos keladigan faylni to'g'ridan-to'g'ri kutubxonaga rekursiv qo'shing.",
"OK": "OK",
"No matching books found in the selected folder.": "Tanlangan papkada mos keladigan kitoblar topilmadi."
"No matching books found in the selected folder.": "Tanlangan papkada mos keladigan kitoblar topilmadi.",
"Send a web article": "Veb-maqola yuborish",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Oqiyotgan maqolangizni kutubxonangizga jonatish uchun Readest brauzer kengaytmasini ornating — u sahifani brauzeringizdan qirqib oladi, shu sababli pul tolov tosigi va faqat hisob bilan kirish talab qilinadigan saytlar ham ishlaydi.",
"Enter a URL starting with http:// or https://": "http:// yoki https:// bilan boshlanadigan URL kiriting",
"Import": "Import qilish",
"Import from Web URL": "Veb URLdan import qilish",
"From Web URL": "Veb URLdan",
"Paste an article link. Readest clips the page and saves it to your library.": "Maqola havolasini joylashtiring. Readest sahifani olib, kutubxonangizga saqlaydi.",
"Saving to your Readest library…": "Readest kutubxonangizga saqlanmoqda…",
"Saving to Readest": "Readest'ga saqlanmoqda",
"Loading article…": "Maqola yuklanmoqda…",
"Capturing article…": "Maqola olinmoqda…",
"Saved to Readest": "Readest'ga saqlandi"
}
@@ -1481,5 +1481,17 @@
"Import all into library": "Nhập tất cả vào thư viện",
"Recursively add every matching file directly to the library.": "Đệ quy thêm mọi tệp khớp trực tiếp vào thư viện.",
"OK": "OK",
"No matching books found in the selected folder.": "Không tìm thấy sách khớp trong thư mục đã chọn."
"No matching books found in the selected folder.": "Không tìm thấy sách khớp trong thư mục đã chọn.",
"Send a web article": "Gửi một bài viết web",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "Cài tiện ích trình duyệt Readest để gửi bài viết bạn đang đọc vào thư viện — nó cắt trang từ trình duyệt của bạn nên các trang trả phí và yêu cầu đăng nhập cũng hoạt động.",
"Enter a URL starting with http:// or https://": "Nhập URL bắt đầu bằng http:// hoặc https://",
"Import": "Nhập",
"Import from Web URL": "Nhập từ URL web",
"From Web URL": "Từ URL web",
"Paste an article link. Readest clips the page and saves it to your library.": "Dán liên kết bài viết. Readest sẽ chụp trang và lưu vào thư viện của bạn.",
"Saving to your Readest library…": "Đang lưu vào thư viện Readest của bạn…",
"Saving to Readest": "Đang lưu vào Readest",
"Loading article…": "Đang tải bài viết…",
"Capturing article…": "Đang chụp bài viết…",
"Saved to Readest": "Đã lưu vào Readest"
}
@@ -1481,5 +1481,17 @@
"Import all into library": "全部导入到书库",
"Recursively add every matching file directly to the library.": "递归地将所有匹配的文件直接添加到书库。",
"OK": "确定",
"No matching books found in the selected folder.": "在所选文件夹中未找到匹配的书籍。"
"No matching books found in the selected folder.": "在所选文件夹中未找到匹配的书籍。",
"Send a web article": "发送网页文章",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "安装 Readest 浏览器扩展,把你正在阅读的文章发送到你的书库——它直接在你的浏览器中剪辑页面,因此付费墙和需要登录的网站也能正常使用。",
"Enter a URL starting with http:// or https://": "请输入以 http:// 或 https:// 开头的网址",
"Import": "导入",
"Import from Web URL": "从网页 URL 导入",
"From Web URL": "从网页 URL",
"Paste an article link. Readest clips the page and saves it to your library.": "粘贴文章链接。Readest 会抓取页面并保存到您的书库。",
"Saving to your Readest library…": "正在保存到您的 Readest 书库…",
"Saving to Readest": "正在保存到 Readest",
"Loading article…": "正在加载文章…",
"Capturing article…": "正在抓取文章…",
"Saved to Readest": "已保存到 Readest"
}
@@ -1481,5 +1481,17 @@
"Import all into library": "全部匯入書庫",
"Recursively add every matching file directly to the library.": "遞迴地將所有相符的檔案直接新增到書庫。",
"OK": "確定",
"No matching books found in the selected folder.": "在所選資料夾中找不到相符的書籍。"
"No matching books found in the selected folder.": "在所選資料夾中找不到相符的書籍。",
"Send a web article": "傳送網頁文章",
"Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.": "安裝 Readest 瀏覽器擴充功能,把你正在閱讀的文章傳送到你的書庫——它直接在你的瀏覽器中剪輯頁面,因此付費牆和需要登入的網站也能正常使用。",
"Enter a URL starting with http:// or https://": "請輸入以 http:// 或 https:// 開頭的網址",
"Import": "匯入",
"Import from Web URL": "從網頁 URL 匯入",
"From Web URL": "從網頁 URL",
"Paste an article link. Readest clips the page and saves it to your library.": "貼上文章連結。Readest 會擷取頁面並儲存到您的書庫。",
"Saving to your Readest library…": "正在儲存到您的 Readest 書庫…",
"Saving to Readest": "正在儲存到 Readest",
"Loading article…": "正在載入文章…",
"Capturing article…": "正在擷取文章…",
"Saved to Readest": "已儲存到 Readest"
}
+645
View File
@@ -0,0 +1,645 @@
// Spawn a hidden Tauri webview that loads the target URL with the real
// browser engine (WebKit2GTK / WKWebView / WebView2), wait for the page to
// render including JS, and stream the rendered `outerHTML` back to the
// caller. Solves the Cloudflare / Medium / paywall case the Rust HTTP
// client cannot: a real browser carries the correct TLS fingerprint and
// runs the page's own scripts, so bot challenges resolve naturally.
//
// Bridge from webview → Rust:
//
// The first attempt used a custom `readest-clip://` URI scheme + `fetch`.
// WebKit treats custom (non-https) schemes as *insecure content* when
// called from an https origin and blocks them — that's not a CSP rule we
// can relax. Browsers DO treat `http://127.0.0.1` as a potentially-
// trustworthy origin (no mixed-content block from https), so we spin up
// a one-shot localhost HTTP server per clip and the init script POSTs
// the outerHTML to it. Same pattern `tauri-plugin-oauth` uses.
//
// Wire shape:
//
// [JS] [Rust] [hidden webview]
// invoke('clip_url', url) ─┬─▶ bind 127.0.0.1:RANDOM_PORT
// │
// ├─▶ WebviewWindowBuilder::External(url)
// │ + initialization_script(port, token)
// │
// │ (page loads, JS runs)
// │
// │ ◀─── fetch('http://127.0.0.1:{port}/clip/{token}',
// │ { method: 'POST', body: outerHTML })
// │
// │ the tokio listener accepts, parses the
// │ request, sends body via oneshot
// │
// ▼
// ◀── outerHTML close webview, return HTML
use std::time::Duration;
use serde::Deserialize;
#[cfg(target_os = "macos")]
use tauri::TitleBarStyle;
use tauri::{AppHandle, Url, WebviewUrl, WebviewWindowBuilder};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpListener;
use tokio::sync::oneshot;
/// Localised strings and theme colours supplied by the JS caller. Defaults
/// are English / Readest's dark palette so a caller that omits a field
/// (tests, future Rust-only callers) still gets readable text and chrome.
#[derive(Deserialize, Default)]
#[serde(rename_all = "camelCase", default)]
pub struct ClipOptions {
window_title: Option<String>,
overlay_title: Option<String>,
loading_status: Option<String>,
capturing_status: Option<String>,
saved_title: Option<String>,
/// `#rrggbb` — matches `themeCode.bg` (base-100) in the renderer.
background: Option<String>,
/// `#rrggbb` — matches `themeCode.fg` (base-content) in the renderer.
foreground: Option<String>,
}
impl ClipOptions {
fn window_title(&self) -> &str {
self.window_title
.as_deref()
.unwrap_or("Saving to your Readest library…")
}
fn overlay_title(&self) -> &str {
self.overlay_title.as_deref().unwrap_or("Saving to Readest")
}
fn loading_status(&self) -> &str {
self.loading_status.as_deref().unwrap_or("Loading article…")
}
fn capturing_status(&self) -> &str {
self.capturing_status
.as_deref()
.unwrap_or("Capturing article…")
}
fn saved_title(&self) -> &str {
self.saved_title.as_deref().unwrap_or("Saved to Readest")
}
fn background(&self) -> &str {
self.background.as_deref().unwrap_or("#1f2024")
}
fn foreground(&self) -> &str {
self.foreground.as_deref().unwrap_or("#f5f5f7")
}
}
/// Parse a `#rrggbb` colour string into 8-bit RGB components. Returns
/// `None` for any malformed input — the caller falls back to whatever
/// default it had.
fn parse_hex_color(s: &str) -> Option<(u8, u8, u8)> {
let hex = s.trim().trim_start_matches('#');
if hex.len() != 6 {
return None;
}
let r = u8::from_str_radix(&hex[0..2], 16).ok()?;
let g = u8::from_str_radix(&hex[2..4], 16).ok()?;
let b = u8::from_str_radix(&hex[4..6], 16).ok()?;
Some((r, g, b))
}
/// HTML-escape a translated string before inlining it into the bridge
/// page or the loading overlay's static markup. JS string literals use
/// `serde_json::to_string` (which already escapes correctly for JS).
fn escape_html(s: &str) -> String {
s.replace('&', "&amp;")
.replace('<', "&lt;")
.replace('>', "&gt;")
.replace('"', "&quot;")
.replace('\'', "&#39;")
}
/// Monotonic + nanosecond timestamp token — unique enough; the token is
/// not a security boundary on its own (the listener only binds to
/// 127.0.0.1 and we close it after the first valid POST), but it makes
/// the URL path predictable for debugging and prevents a rogue process
/// on the loopback interface from accidentally hitting us.
fn next_token() -> String {
use std::sync::atomic::{AtomicU64, Ordering};
static COUNTER: AtomicU64 = AtomicU64::new(0);
let n = COUNTER.fetch_add(1, Ordering::Relaxed);
let ts = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_nanos();
format!("{:x}{:x}", ts, n)
}
// The request URL now carries the page HTML as base64, so the request
// LINE alone can be megabytes — bump generously.
const MAX_REQUEST_BYTES: usize = 64 * 1024 * 1024;
const READ_CHUNK_BYTES: usize = 64 * 1024;
const SOCKET_TIMEOUT: Duration = Duration::from_secs(10);
/// Find the `\r\n\r\n` that terminates the HTTP request headers.
fn find_header_end(buf: &[u8]) -> Option<usize> {
buf.windows(4).position(|w| w == b"\r\n\r\n")
}
/// Pull `Content-Length` out of header bytes (case-insensitive).
fn parse_content_length(headers: &str) -> usize {
for line in headers.split("\r\n").skip(1) {
if let Some((name, value)) = line.split_once(':') {
if name.eq_ignore_ascii_case("content-length") {
return value.trim().parse().unwrap_or(0);
}
}
}
0
}
/// Capture loop. The clip webview navigates to
/// `GET /clip/{token}?d={base64-HTML}` — top-level navigation isn't
/// governed by CSP `connect-src` / `form-action`, and the URL itself
/// carries the data (so we don't need any cross-origin storage trick).
/// Server decodes the base64, signals the oneshot, returns a tiny
/// "captured" page so the user can see the round-trip worked.
async fn capture_one(
listener: TcpListener,
token: String,
tx: oneshot::Sender<String>,
saved_title: String,
background: String,
foreground: String,
) {
let mut tx = Some(tx);
let expected_prefix = format!("/clip/{}", token);
let saved_title_safe = escape_html(&saved_title);
// CSS-context escape: the caller-provided colour goes into a
// `style="…"` attribute. Reuse the HTML escape so any quote /
// angle-bracket can't break out of the attribute or smuggle markup.
let bg_css = escape_html(&background);
let fg_css = escape_html(&foreground);
loop {
let Ok((mut stream, _peer)) = listener.accept().await else {
break;
};
let mut buf = Vec::with_capacity(READ_CHUNK_BYTES);
let mut chunk = vec![0u8; READ_CHUNK_BYTES];
let mut header_end: Option<usize> = None;
let mut content_length: usize = 0;
loop {
if buf.len() > MAX_REQUEST_BYTES {
break;
}
let read = tokio::time::timeout(SOCKET_TIMEOUT, stream.read(&mut chunk)).await;
let n = match read {
Ok(Ok(n)) if n > 0 => n,
_ => break,
};
buf.extend_from_slice(&chunk[..n]);
if header_end.is_none() {
if let Some(idx) = find_header_end(&buf) {
header_end = Some(idx);
let headers_str = std::str::from_utf8(&buf[..idx]).unwrap_or("");
content_length = parse_content_length(headers_str);
}
}
if let Some(idx) = header_end {
if buf.len() >= idx + 4 + content_length {
break;
}
}
}
let Some(hdr_end) = header_end else {
continue;
};
let headers_str = std::str::from_utf8(&buf[..hdr_end]).unwrap_or("");
let first_line = headers_str.lines().next().unwrap_or("");
let mut parts = first_line.split_whitespace();
let method = parts.next().unwrap_or("");
let target = parts.next().unwrap_or("");
// `target` is the request-target — `/clip/{token}?d=...`. Split
// path vs query.
let (path, query) = match target.find('?') {
Some(i) => (&target[..i], &target[i + 1..]),
None => (target, ""),
};
if method != "GET" || path != expected_prefix {
let _ = stream
.write_all(b"HTTP/1.1 404 Not Found\r\nContent-Length: 0\r\n\r\n")
.await;
continue;
}
// Decode `d=<base64>` out of the query string.
let mut data_b64: Option<&str> = None;
for pair in query.split('&') {
if let Some(v) = pair.strip_prefix("d=") {
data_b64 = Some(v);
break;
}
}
let html = match data_b64.and_then(decode_b64) {
Some(s) => s,
None => {
let body = b"capture: missing or invalid `d` query param";
let mut response = Vec::with_capacity(128 + body.len());
response.extend_from_slice(b"HTTP/1.1 400 Bad Request\r\n");
response.extend_from_slice(b"Content-Type: text/plain; charset=utf-8\r\n");
response.extend_from_slice(
format!("Content-Length: {}\r\n\r\n", body.len()).as_bytes(),
);
response.extend_from_slice(body);
let _ = stream.write_all(&response).await;
continue;
}
};
// Tell the user / devtools the round-trip succeeded with the
// same look as the loading overlay — same dark background,
// checkmark instead of spinner. Window closes a moment later.
let confirmation = format!(
r##"<!DOCTYPE html><html><head><meta charset="utf-8"><title>{title}</title></head>
<body style="margin:0;height:100vh;background:{bg};color:{fg};
font-family:-apple-system,BlinkMacSystemFont,&quot;Segoe UI&quot;,Roboto,sans-serif;
display:flex;flex-direction:column;align-items:center;justify-content:center;gap:14px;
padding:24px;box-sizing:border-box;text-align:center">
<div style="width:36px;height:36px;border-radius:50%;background:rgba(76,175,80,0.18);
display:flex;align-items:center;justify-content:center">
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="#7cd47e"
stroke-width="3" stroke-linecap="round" stroke-linejoin="round"><polyline points="20 6 9 17 4 12"/></svg>
</div>
<div style="font-size:15px;font-weight:600">{title}</div>
</body></html>"##,
title = saved_title_safe,
bg = bg_css,
fg = fg_css,
);
// bytes count was diagnostic-only; dropped from the user-facing
// page so the captured-state stays clean across locales.
let _ = html.len();
let mut response = Vec::with_capacity(256 + confirmation.len());
response.extend_from_slice(b"HTTP/1.1 200 OK\r\n");
response.extend_from_slice(b"Content-Type: text/html; charset=utf-8\r\n");
response.extend_from_slice(
format!("Content-Length: {}\r\n\r\n", confirmation.len()).as_bytes(),
);
response.extend_from_slice(confirmation.as_bytes());
let _ = stream.write_all(&response).await;
if let Some(tx) = tx.take() {
let _ = tx.send(html);
}
break;
}
}
/// Decode a URL-safe base64 string (the JS side uses `btoa` which
/// produces standard base64; we also accept URL-safe variants in case
/// a future caller swaps). Returns the decoded UTF-8 string, or None
/// on any decode error.
fn decode_b64(s: &str) -> Option<String> {
use std::collections::HashMap;
// Tiny hand-rolled base64 decoder — avoids pulling in another
// crate for one place. Accepts standard + URL-safe alphabets and
// ignores any non-alphabet character (so `+`/`/` URL-encoded as
// `%2B`/`%2F` would slip through, but we've not URL-encoded the
// body, just escaped it via toString).
static ALPHABET: &str = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/-_";
let table: HashMap<u8, u8> = ALPHABET
.bytes()
.enumerate()
.map(|(i, b)| {
let value = match b {
b'-' => 62,
b'_' => 63,
_ => i as u8,
};
(b, value)
})
.collect();
let mut bytes = Vec::with_capacity(s.len() * 3 / 4);
let mut acc: u32 = 0;
let mut bits = 0;
for &c in s.as_bytes() {
if c == b'=' {
break;
}
let v = match table.get(&c) {
Some(&v) => v as u32,
None => continue, // skip whitespace / unexpected chars
};
acc = (acc << 6) | v;
bits += 6;
if bits >= 8 {
bits -= 8;
bytes.push((acc >> bits) as u8);
acc &= (1u32 << bits) - 1;
}
}
String::from_utf8(bytes).ok()
}
/// Inject a fullscreen loading overlay before the page renders so the
/// user sees a deliberate "Saving…" UI instead of the article flashing
/// by. The overlay is `position:fixed` with the maximum z-index and
/// re-attaches itself for a few hundred milliseconds in case the page's
/// hydration step wipes our node. It's just chrome — the page
/// underneath still loads, runs scripts, and fires its lazy-loaders.
fn loading_overlay_script(
overlay_title: &str,
loading_status: &str,
background: &str,
foreground: &str,
) -> String {
// Inline as JS string literals (JSON encoding handles the escapes).
// `textContent` assignment avoids any HTML injection risk from the
// translated strings themselves; JSON-encoding the colour values
// makes any unexpected character (a stray quote, a CSS expression)
// a syntax error rather than a CSS injection.
let title_json = serde_json::to_string(overlay_title).unwrap_or_else(|_| "\"\"".into());
let status_json = serde_json::to_string(loading_status).unwrap_or_else(|_| "\"\"".into());
let bg_json = serde_json::to_string(background).unwrap_or_else(|_| "\"#1f2024\"".into());
let fg_json = serde_json::to_string(foreground).unwrap_or_else(|_| "\"#f5f5f7\"".into());
format!(
r#"
(function() {{
var TITLE = {title_json};
var STATUS = {status_json};
var BG = {bg_json};
var FG = {fg_json};
function install() {{
if (document.getElementById('__readest_overlay__')) return;
if (!document.documentElement) return;
var ov = document.createElement('div');
ov.id = '__readest_overlay__';
ov.setAttribute('aria-live', 'polite');
ov.style.cssText = [
'position:fixed','inset:0',
'background:' + BG,'color:' + FG,
'font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,sans-serif',
'display:flex','flex-direction:column','align-items:center','justify-content:center',
'gap:14px','padding:24px','box-sizing:border-box','text-align:center',
'z-index:2147483647','pointer-events:auto'
].join(';');
var spin = document.createElement('div');
// Spinner uses the foreground colour with low/high opacity so it
// reads on both light and dark themes.
spin.style.cssText = 'width:36px;height:36px;border:3px solid color-mix(in srgb,' +
' ' + FG + ' 18%, transparent);' +
'border-top-color:color-mix(in srgb,' + FG + ' 85%, transparent);' +
'border-radius:50%;animation:__readest_spin__ 0.8s linear infinite';
var title = document.createElement('div');
title.style.cssText = 'font-size:15px;font-weight:600';
title.textContent = TITLE;
var status = document.createElement('div');
status.id = '__readest_status__';
status.style.cssText = 'font-size:13px;opacity:0.7;max-width:340px;' +
'overflow:hidden;text-overflow:ellipsis;white-space:nowrap';
status.textContent = STATUS;
var style = document.createElement('style');
style.textContent = '@keyframes __readest_spin__{{to{{transform:rotate(360deg)}}}}';
ov.appendChild(spin);
ov.appendChild(title);
ov.appendChild(status);
ov.appendChild(style);
document.documentElement.appendChild(ov);
}}
install();
var attempts = 0;
var iv = setInterval(function() {{
attempts++;
if (attempts > 30 || document.readyState === 'complete') {{
install();
clearInterval(iv);
return;
}}
install();
}}, 200);
window.__readest_setStatus__ = function(text) {{
var el = document.getElementById('__readest_status__');
if (el) el.textContent = text;
}};
}})();
"#,
)
}
/// Hide the usual headless-/automation-flavoured signals before the page's
/// own scripts run. The mask doesn't try to be exhaustive — sites with
/// commercial bot detection (X.com, sophisticated paywalls) will still
/// catch us through canvas / WebGL / audio fingerprinting. The goal is
/// just to clear the "you look like Chrome but `navigator.webdriver` is
/// set" tier of checks.
fn fingerprint_mask_script() -> String {
r#"
(function() {
try {
Object.defineProperty(navigator, 'webdriver', { get: () => undefined });
} catch (e) {}
try {
// Many Chrome-only objects sites probe for.
if (!window.chrome) {
window.chrome = { runtime: {} };
}
} catch (e) {}
try {
// navigator.languages — some checks see an empty list as suspicious.
if (navigator.languages && navigator.languages.length === 0) {
Object.defineProperty(navigator, 'languages', { get: () => ['en-US', 'en'] });
}
} catch (e) {}
})();
"#
.to_string()
}
/// Spawn a hidden webview, load `url`, wait for the rendered HTML, return
/// it. Errors:
/// - "Invalid URL" / "URL must use http or https" — pre-flight validation.
/// - "Could not bind capture port: …" — local listener bind failed.
/// - "Could not create clip webview: …" — Tauri couldn't open the window.
/// - "Page took too long to load" — 30 s timeout elapsed without a POST.
/// - "Webview closed before capture" — the page closed itself, or our
/// `close()` raced the script.
#[cfg(desktop)]
#[tauri::command]
pub async fn clip_url(
app: AppHandle,
url: String,
options: Option<ClipOptions>,
) -> Result<String, String> {
let parsed = Url::parse(&url).map_err(|e| format!("Invalid URL: {}", e))?;
if parsed.scheme() != "http" && parsed.scheme() != "https" {
return Err("URL must use http or https".into());
}
let options = options.unwrap_or_default();
let listener = TcpListener::bind("127.0.0.1:0")
.await
.map_err(|e| format!("Could not bind capture port: {}", e))?;
let port = listener
.local_addr()
.map_err(|e| format!("Could not read capture port: {}", e))?
.port();
let token = next_token();
let (tx, rx) = oneshot::channel::<String>();
let token_for_server = token.clone();
let saved_title_for_server = options.saved_title().to_string();
let bg_for_server = options.background().to_string();
let fg_for_server = options.foreground().to_string();
tokio::spawn(async move {
capture_one(
listener,
token_for_server,
tx,
saved_title_for_server,
bg_for_server,
fg_for_server,
)
.await;
});
let label = format!("clip-{}", token);
let token_json = serde_json::to_string(&token).map_err(|e| e.to_string())?;
let capturing_status_json =
serde_json::to_string(options.capturing_status()).map_err(|e| e.to_string())?;
let init_script = format!(
r#"
(function() {{
console.log('[readest-clip] init script running');
var PORT = {port};
var TOKEN = {token_json};
var CAPTURING_STATUS = {capturing_status_json};
var TARGET = 'http://127.0.0.1:' + PORT + '/clip/' + TOKEN;
var sent = false;
function send(reason) {{
if (sent) return;
sent = true;
try {{
if (window.__readest_setStatus__) {{
window.__readest_setStatus__(CAPTURING_STATUS);
}}
var html = document.documentElement.outerHTML;
console.log('[readest-clip] capturing reason=' + reason +
' bytes=' + html.length);
// Transfer the HTML through the navigation URL itself —
// top-level navigation isn't governed by CSP `connect-src`
// / `form-action`, and WebKit doesn't enforce Private
// Network Access on navigation the way it does on fetch.
// Each earlier transport was blocked by something:
// - fetch / XHR : connect-src + WebKit PNA mixed-content
// - <form action=...> : CSP form-action
// - custom URI scheme : WebKit insecure-content
// - window.name + nav : WebKit clears name on x-origin nav
// unescape(encodeURIComponent(...)) is the canonical
// UTF-8 dance before btoa(), which otherwise throws on
// multi-byte chars (every CJK article).
// URL-safe base64 — replace +/= so the browser doesn't
// percent-encode them and the Rust decoder doesn't have to
// un-encode. Padding stripped.
var b64 = btoa(unescape(encodeURIComponent(html)))
.replace(/\+/g, '-')
.replace(/\//g, '_')
.replace(/=+$/, '');
var sep = TARGET.indexOf('?') >= 0 ? '&' : '?';
window.location.assign(TARGET + sep + 'd=' + b64);
}} catch (e) {{
console.warn('[readest-clip] navigate threw:', e && e.message);
}}
}}
// Capture after the load event + a generous settle so JS
// challenges resolve and IntersectionObserver-based lazy
// loaders fire for content already in the viewport. We used
// to scroll top→bottom to force every lazy image to load,
// but in practice modern sites use a roomy rootMargin and
// most images on the page have already started loading by
// the time we hit this point.
window.addEventListener('load', function() {{
setTimeout(function() {{ send('load+settle'); }}, 3000);
}}, {{ once: true }});
// Hard fallback in case `load` never fires (SPA, error state,
// long-running redirect chain).
setTimeout(function() {{ send('hard-timeout'); }}, 20000);
}})();
"#,
);
// Send a real Chrome UA. Tauri's default UA reports Safari on macOS
// and Edge/WebView2 on Windows; sites with aggressive bot detection
// (X / Twitter, some news sites) cross-check the UA against
// navigator.* fingerprints and reject the mismatch.
const BROWSER_UA: &str = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 \
(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36";
// macOS doesn't honour `.visible(false)` for a WKWebView that needs
// its JS timers to keep firing — the public Tauri API can't reach
// the private NSWindow flags that would hide it without freezing
// scripts. The window IS going to be on screen briefly. Match the
// chrome style Readest's main/reader windows use so it doesn't read
// as a foreign popup: on macOS the standard window frame with an
// overlay (transparent) title bar; on other desktops, decorationless
// with a drop shadow. The loading overlay (injected via initialization
// script) covers the article render so the user sees a deliberate
// "Saving…" state rather than the article flashing by.
let win_builder = WebviewWindowBuilder::new(&app, &label, WebviewUrl::External(parsed))
.title(options.window_title())
.visible(true)
.center()
.resizable(false)
.inner_size(640.0, 480.0)
.user_agent(BROWSER_UA)
.initialization_script(fingerprint_mask_script())
.initialization_script(loading_overlay_script(
options.overlay_title(),
options.loading_status(),
options.background(),
options.foreground(),
))
.initialization_script(&init_script);
// Tint the window's native background to the caller's theme `bg` so
// the brief flash before the loading overlay attaches (and any sliver
// around the WKWebView during resize/inset adjustments) matches the
// main window's palette instead of flashing white.
let win_builder = if let Some((r, g, b)) = parse_hex_color(options.background()) {
win_builder.background_color(tauri::window::Color(r, g, b, 255))
} else {
win_builder
};
#[cfg(target_os = "macos")]
let win_builder = win_builder
.decorations(true)
.title_bar_style(TitleBarStyle::Overlay);
#[cfg(all(not(target_os = "macos"), desktop))]
let win_builder = win_builder.decorations(false).shadow(true);
let webview_result = win_builder.build();
let webview = match webview_result {
Ok(w) => w,
Err(e) => return Err(format!("Could not create clip webview: {}", e)),
};
// 30 s covers a slow page load and a Cloudflare-style JS challenge
// (515 s on bad networks) with margin for the settle delay.
let result = tokio::time::timeout(Duration::from_secs(30), rx).await;
// Always close the clip window after capture (or timeout) — the
// window flashing on screen for a few seconds is the brief mode
// we want, not a lingering "Saving…" window the user has to close
// themselves.
let _ = webview.close();
match result {
Ok(Ok(html)) => Ok(html),
Ok(Err(_)) => Err("Webview closed before capture".into()),
Err(_) => Err("Page took too long to load".into()),
}
}
+4
View File
@@ -22,6 +22,8 @@ use tauri_plugin_fs::FsExt;
#[cfg(desktop)]
use tauri::{Listener, Url};
#[cfg(desktop)]
mod clip_url;
mod dir_scanner;
#[cfg(any(target_os = "macos", target_os = "windows", target_os = "linux"))]
mod discord_rpc;
@@ -233,6 +235,8 @@ pub fn run() {
discord_rpc::update_book_presence,
#[cfg(any(target_os = "macos", target_os = "windows", target_os = "linux"))]
discord_rpc::clear_book_presence,
#[cfg(desktop)]
clip_url::clip_url,
])
.plugin(tauri_plugin_fs::init())
.plugin(tauri_plugin_persisted_scope::init())
@@ -93,6 +93,17 @@ pub fn setup_traffic_light_positioner<R: Runtime>(window: Window<R>) {
use objc::runtime::{Object, Sel};
use std::ffi::c_void;
// The on_window_did_resize handler below already gates positioning to
// the main/reader windows by label — extending the same gate to the
// initial positioning. Other windows (the clip-* webview, future
// decorationless utility windows) have no standard NSWindow buttons,
// and `close.superview().superview()` in position_traffic_lights
// would null-deref on them.
let label = window.label().to_string();
if label != "main" && !label.starts_with("reader") {
return;
}
// Do the initial positioning
unsafe {
position_traffic_lights(
@@ -0,0 +1,166 @@
import { describe, expect, test, beforeEach, vi, afterEach } from 'vitest';
import { bundleAssets } from '@/services/send/conversion/assetBundler';
// A tiny 1×1 transparent PNG so the bundler has real bytes to hash.
const PNG = new Uint8Array([
0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, 0x52,
0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x08, 0x06, 0x00, 0x00, 0x00, 0x1f, 0x15, 0xc4,
0x89, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x44, 0x41, 0x54, 0x78, 0x9c, 0x62, 0x00, 0x01, 0x00, 0x00,
0x05, 0x00, 0x01, 0x0d, 0x0a, 0x2d, 0xb4, 0x00, 0x00, 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae,
0x42, 0x60, 0x82,
]);
function pngResponse(): Response {
return new Response(PNG, {
status: 200,
headers: { 'content-type': 'image/png' },
});
}
describe('bundleAssets', () => {
let fetchSpy: ReturnType<typeof vi.spyOn>;
beforeEach(() => {
fetchSpy = vi.spyOn(globalThis, 'fetch').mockImplementation(async () => pngResponse());
});
afterEach(() => {
fetchSpy.mockRestore();
});
test('resolves data-src lazy-loading attrs', async () => {
// Some sites ship `<img data-src="..." src="">` and lazy-load on
// scroll. The bundler must pick up `data-src` rather than the empty
// `src`.
const html = `<p>hello</p><img data-src="https://cdn.example.com/lazy.png" alt="cover">`;
const result = await bundleAssets(html, 'https://example.com/article');
expect(fetchSpy).toHaveBeenCalledWith(
'https://cdn.example.com/lazy.png',
expect.objectContaining({ redirect: 'follow' }),
);
expect(result.images).toHaveLength(1);
expect(result.images[0]!.mime).toBe('image/png');
expect(result.html).toContain('src="images/');
expect(result.html).not.toContain('data-src');
expect(result.missing).toBe(0);
});
test('prefers srcset over data-* lazy attrs (modern lazy-loading sites)', async () => {
// Medium / Substack / NYT put the real responsive image in srcset and
// leave src/data-src as a tiny placeholder. srcset wins.
const html = `<img data-original="https://cdn.example.com/placeholder.png" srcset="https://cdn.example.com/2x.png 2x">`;
const result = await bundleAssets(html, 'https://example.com');
expect(fetchSpy).toHaveBeenCalledWith(
'https://cdn.example.com/2x.png',
expect.objectContaining({ redirect: 'follow' }),
);
expect(result.images).toHaveLength(1);
});
test('picks the largest-resolution variant from a srcset (not the first)', async () => {
const html = `<img srcset="https://cdn.example.com/sm.png 480w, https://cdn.example.com/lg.png 1024w">`;
const result = await bundleAssets(html, 'https://example.com');
// 1024w wins over 480w — full quality in the EPUB, not the thumbnail.
expect(fetchSpy).toHaveBeenCalledWith(
'https://cdn.example.com/lg.png',
expect.objectContaining({ redirect: 'follow' }),
);
expect(result.images).toHaveLength(1);
});
test('prefers srcset over a tiny placeholder in src', async () => {
// Real-world Medium shape: src is a 60-pixel-wide LQIP for lazy
// loading; srcset has the real responsive variants.
const html = `<img src="https://cdn.example.com/lqip-60.jpg" srcset="https://cdn.example.com/v-700.jpg 700w, https://cdn.example.com/v-1400.jpg 1400w">`;
const result = await bundleAssets(html, 'https://example.com');
expect(fetchSpy).toHaveBeenCalledWith(
'https://cdn.example.com/v-1400.jpg',
expect.objectContaining({ redirect: 'follow' }),
);
expect(result.images).toHaveLength(1);
});
test('dedupes the same URL referenced twice', async () => {
const html = `
<img src="https://cdn.example.com/hero.png">
<p>some text</p>
<img src="https://cdn.example.com/hero.png">
`;
const result = await bundleAssets(html, 'https://example.com');
expect(fetchSpy).toHaveBeenCalledTimes(1);
expect(result.images).toHaveLength(1);
});
test('drops <iframe> / <video> / <audio> / <embed> / <object>', async () => {
const html = `
<p>article</p>
<iframe src="https://example.com/embed"></iframe>
<video src="https://example.com/v.mp4"></video>
<audio src="https://example.com/a.mp3"></audio>
<embed src="https://example.com/e.swf">
<object data="https://example.com/o.pdf"></object>
`;
const result = await bundleAssets(html, 'https://example.com');
expect(result.html).not.toMatch(/<iframe|<video|<audio|<embed|<object/);
expect(fetchSpy).not.toHaveBeenCalled();
});
test('keeps inline <svg> in place without fetching', async () => {
const html = `<svg viewBox="0 0 10 10"><rect width="10" height="10" fill="red"/></svg>`;
const result = await bundleAssets(html, 'https://example.com');
expect(result.html).toContain('<svg');
expect(result.html).toContain('<rect');
expect(fetchSpy).not.toHaveBeenCalled();
});
test('flattens <picture> to a single <img> pointing at the chosen source', async () => {
const html = `
<picture>
<source srcset="https://cdn.example.com/x.webp" type="image/webp">
<img src="https://cdn.example.com/x.jpg" alt="hero">
</picture>
`;
const result = await bundleAssets(html, 'https://example.com');
// The webp source wins because it's first.
expect(fetchSpy).toHaveBeenCalledWith(
'https://cdn.example.com/x.webp',
expect.objectContaining({ redirect: 'follow' }),
);
expect(result.html).not.toContain('<picture');
expect(result.html).toContain('<img');
expect(result.html).toContain('alt="hero"');
});
test('counts a 404 as missing, leaves the <img> with no src so alt-text shows', async () => {
fetchSpy.mockImplementationOnce(async () => new Response('not found', { status: 404 }));
const html = `<img src="https://cdn.example.com/gone.png" alt="missing">`;
const result = await bundleAssets(html, 'https://example.com');
expect(result.missing).toBe(1);
expect(result.images).toHaveLength(0);
expect(result.html).not.toMatch(/src="https/);
expect(result.html).toContain('alt="missing"');
});
test('resolves relative URLs against the page URL', async () => {
const html = `<img src="/imgs/local.png">`;
await bundleAssets(html, 'https://news.example.com/articles/123');
expect(fetchSpy).toHaveBeenCalledWith(
'https://news.example.com/imgs/local.png',
expect.objectContaining({ redirect: 'follow' }),
);
});
test('strips loading/decoding/fetchpriority and srcset noise from kept <img>', async () => {
const html = `<img src="https://cdn.example.com/a.png" loading="lazy" decoding="async" fetchpriority="low" srcset="https://cdn.example.com/2x.png 2x">`;
const result = await bundleAssets(html, 'https://example.com');
expect(result.html).not.toMatch(/loading|decoding|fetchpriority|srcset/);
});
test('drops 1×1 tracking pixels without fetching', async () => {
const html = `<img src="https://tracker.example.com/p.gif" width="1" height="1">`;
const result = await bundleAssets(html, 'https://example.com');
expect(fetchSpy).not.toHaveBeenCalled();
expect(result.images).toHaveLength(0);
});
});
@@ -87,3 +87,22 @@ describe('convertToEpub — article', () => {
expect(book.file.size).toBeGreaterThan(0);
});
});
describe('convertToEpub — page (quality floor)', () => {
test('rejects bot-detection / verification-page output (extracted text too short)', async () => {
// Bot-detection / verification screen: just enough markup for
// Readability to find a sliver of content, but well under the 400-
// char quality floor.
const html = `<html><head><title></title></head><body>
<h1>Verify you are human</h1>
<p>Please complete the check to continue.</p>
<button>Verify</button>
</body></html>`;
await expect(
convertToEpub({ kind: 'page', html, url: 'https://example.com/article' }),
).rejects.toThrow(ConversionError);
await expect(
convertToEpub({ kind: 'page', html, url: 'https://example.com/article' }),
).rejects.toThrow(/verification screen|login wall/i);
});
});
@@ -0,0 +1,129 @@
import { describe, expect, test } from 'vitest';
import { buildNavMap, extractHeadings } from '@/services/send/conversion/toc';
const ESC = (s: string) => s.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/"/g, '&quot;');
describe('extractHeadings', () => {
test('returns h1h6 in document order with their level', () => {
const { headings } = extractHeadings(`
<h1>Intro</h1>
<p>...</p>
<h2>Setup</h2>
<h3>Install</h3>
<h2>Usage</h2>
`);
expect(headings.map((h) => [h.level, h.text])).toEqual([
[1, 'Intro'],
[2, 'Setup'],
[3, 'Install'],
[2, 'Usage'],
]);
});
test('slugifies heading text into ids and writes them onto the elements', () => {
const { html, headings } = extractHeadings(`<h2>Hello, world!</h2>`);
expect(headings[0]!.id).toBe('hello-world');
expect(html).toContain('id="hello-world"');
});
test('preserves an existing id verbatim instead of slugging', () => {
const { headings, html } = extractHeadings(`<h2 id="custom-anchor">Section</h2>`);
expect(headings[0]!.id).toBe('custom-anchor');
expect(html).toContain('id="custom-anchor"');
});
test('de-duplicates colliding slugs with -2, -3, …', () => {
const { headings, html } = extractHeadings(`
<h2>Intro</h2>
<h2>Intro</h2>
<h2>Intro</h2>
`);
expect(headings.map((h) => h.id)).toEqual(['intro', 'intro-2', 'intro-3']);
expect((html.match(/id="intro"/g) || []).length).toBe(1);
expect((html.match(/id="intro-2"/g) || []).length).toBe(1);
expect((html.match(/id="intro-3"/g) || []).length).toBe(1);
});
test('falls back to h-N when the heading text strips to nothing (CJK, emoji)', () => {
const { headings } = extractHeadings(`<h2>简介</h2><h2>🚀</h2>`);
expect(headings[0]!.id).toBe('h-1');
expect(headings[1]!.id).toBe('h-2');
});
test('skips empty headings entirely (no id, no toc entry)', () => {
const { headings } = extractHeadings(`<h2></h2><h2> </h2><h2>Real</h2>`);
expect(headings).toHaveLength(1);
expect(headings[0]!.text).toBe('Real');
});
});
describe('buildNavMap', () => {
test('nests deeper levels under shallower ones', () => {
const xml = buildNavMap(
[
{ id: 'intro', text: 'Intro', level: 1 },
{ id: 'setup', text: 'Setup', level: 2 },
{ id: 'install', text: 'Install', level: 3 },
{ id: 'usage', text: 'Usage', level: 2 },
],
'OEBPS/chapter1.xhtml',
ESC,
);
// Intro contains Setup, Setup contains Install, then Usage is a sibling of Setup.
expect(xml).toContain('<navPoint id="np1"');
expect(xml).toContain('<navPoint id="np2"');
expect(xml).toContain('<navPoint id="np3"');
expect(xml).toContain('<navPoint id="np4"');
// Install is closed before Usage opens (sibling of Setup).
const installIdx = xml.indexOf('install');
const usageIdx = xml.indexOf('usage');
expect(installIdx).toBeLessThan(usageIdx);
const installClose = xml.indexOf('</navPoint>', installIdx);
expect(installClose).toBeLessThan(usageIdx);
});
test('uses sequential playOrder starting at 1', () => {
const xml = buildNavMap(
[
{ id: 'a', text: 'A', level: 2 },
{ id: 'b', text: 'B', level: 2 },
{ id: 'c', text: 'C', level: 2 },
],
'OEBPS/chapter1.xhtml',
ESC,
);
expect(xml).toContain('playOrder="1"');
expect(xml).toContain('playOrder="2"');
expect(xml).toContain('playOrder="3"');
expect(xml).not.toContain('playOrder="4"');
});
test('emits the chapter#fragment link for each entry', () => {
const xml = buildNavMap(
[{ id: 'intro', text: 'Intro', level: 2 }],
'OEBPS/chapter1.xhtml',
ESC,
);
expect(xml).toContain('<content src="OEBPS/chapter1.xhtml#intro"/>');
});
test('returns empty string when given no headings', () => {
expect(buildNavMap([], 'OEBPS/chapter1.xhtml', ESC)).toBe('');
});
test('balances opens and closes so the navMap parses', () => {
const xml = buildNavMap(
[
{ id: 'a', text: 'A', level: 1 },
{ id: 'b', text: 'B', level: 3 }, // skip level 2
{ id: 'c', text: 'C', level: 2 },
{ id: 'd', text: 'D', level: 4 },
],
'OEBPS/chapter1.xhtml',
ESC,
);
const opens = (xml.match(/<navPoint/g) || []).length;
const closes = (xml.match(/<\/navPoint>/g) || []).length;
expect(opens).toBe(closes);
});
});
@@ -0,0 +1,115 @@
'use client';
import React, { useEffect, useState } from 'react';
import { MdLink } from 'react-icons/md';
import Dialog from '@/components/Dialog';
import { useTranslation } from '@/hooks/useTranslation';
interface ImportFromUrlDialogProps {
isOpen: boolean;
onClose: () => void;
onSubmit: (url: string) => Promise<void>;
}
/**
* Modal for the Library import-menu's "From URL" entry. Collects an article
* URL, hands it to the page-clip pipeline, and closes. Tauri-only entry point
* a web build can't fetch cross-origin pages so this dialog never mounts
* there.
*/
const ImportFromUrlDialog: React.FC<ImportFromUrlDialogProps> = ({ isOpen, onClose, onSubmit }) => {
const _ = useTranslation();
const [url, setUrl] = useState('');
const [submitting, setSubmitting] = useState(false);
const [error, setError] = useState<string | null>(null);
// Reset transient state every time the dialog reopens.
useEffect(() => {
if (!isOpen) return;
setUrl('');
setSubmitting(false);
setError(null);
}, [isOpen]);
const submit = async () => {
const target = url.trim();
if (!/^https?:\/\//i.test(target)) {
setError(_('Enter a URL starting with http:// or https://'));
return;
}
setSubmitting(true);
setError(null);
try {
await onSubmit(target);
onClose();
} catch (e) {
// Tauri's `invoke` rejects with the raw Err string from Rust (not
// wrapped in Error), and our pipeline also throws Error objects —
// surface either shape directly so the user sees the real cause.
const message =
e instanceof Error ? e.message : typeof e === 'string' ? e : _('Could not fetch this page');
setError(message);
setSubmitting(false);
}
};
return (
<Dialog
isOpen={isOpen}
onClose={onClose}
title={_('Import from Web URL')}
// Size to content: `sm:!h-auto` overrides the Dialog default of
// `sm:h-[65%]` so we don't end up with a near-full-height modal
// for a one-line form. Width-constrained to a comfortable
// reading measure.
boxClassName='sm:!w-[480px] sm:!max-w-[480px] sm:!h-auto sm:!max-h-[80vh]'
>
<div className='flex flex-col gap-4 pb-6 pt-2'>
<p className='text-base-content/60 text-sm leading-relaxed'>
{_('Paste an article link. Readest clips the page and saves it to your library.')}
</p>
<input
type='url'
autoFocus
// Explicit placeholder colour — daisyUI's `input-bordered`
// leaves placeholders too dark on light themes; the user
// can mistake the example for actual content.
className='input input-bordered eink-bordered placeholder:text-base-content/35 w-full'
placeholder='https://example.com/article'
value={url}
disabled={submitting}
onChange={(e) => setUrl(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') void submit();
}}
/>
{error && <p className='text-error text-sm leading-relaxed'>{error}</p>}
<div className='flex justify-end gap-2 pt-1'>
<button
type='button'
className='btn btn-ghost btn-sm eink-bordered'
onClick={onClose}
disabled={submitting}
>
{_('Cancel')}
</button>
<button
type='button'
className='btn btn-contrast btn-sm'
onClick={() => void submit()}
disabled={submitting || !url.trim()}
>
{submitting ? (
<span className='loading loading-spinner loading-xs' />
) : (
<MdLink className='h-4 w-4' />
)}
{_('Import')}
</button>
</div>
</div>
</Dialog>
);
};
export default ImportFromUrlDialog;
@@ -1,5 +1,5 @@
import clsx from 'clsx';
import { MdRssFeed } from 'react-icons/md';
import { MdLink, MdRssFeed } from 'react-icons/md';
import { IoFileTray } from 'react-icons/io5';
import { useEnv } from '@/context/EnvContext';
import { useTranslation } from '@/hooks/useTranslation';
@@ -10,6 +10,7 @@ interface ImportMenuProps {
setIsDropdownOpen?: (open: boolean) => void;
onImportBooksFromFiles: () => void;
onImportBooksFromDirectory?: () => void;
onImportBookFromUrl?: () => void;
onOpenCatalogManager: () => void;
}
@@ -17,6 +18,7 @@ const ImportMenu: React.FC<ImportMenuProps> = ({
setIsDropdownOpen,
onImportBooksFromFiles,
onImportBooksFromDirectory,
onImportBookFromUrl,
onOpenCatalogManager,
}) => {
const _ = useTranslation();
@@ -32,6 +34,11 @@ const ImportMenu: React.FC<ImportMenuProps> = ({
setIsDropdownOpen?.(false);
};
const handleImportFromUrl = () => {
onImportBookFromUrl?.();
setIsDropdownOpen?.(false);
};
const handleOpenCatalogManager = () => {
onOpenCatalogManager();
setIsDropdownOpen?.(false);
@@ -54,6 +61,13 @@ const ImportMenu: React.FC<ImportMenuProps> = ({
onClick={handleImportFromDirectory}
/>
)}
{onImportBookFromUrl && (
<MenuItem
label={_('From Web URL')}
Icon={<MdLink className='h-5 w-5' />}
onClick={handleImportFromUrl}
/>
)}
<MenuItem
label={appService?.isOnlineCatalogsAccessible ? _('Online Library') : _('OPDS Catalogs')}
Icon={<MdRssFeed className='h-5 w-5' />}
@@ -28,6 +28,7 @@ interface LibraryHeaderProps {
onPullLibrary: () => void;
onImportBooksFromFiles: () => void;
onImportBooksFromDirectory?: () => void;
onImportBookFromUrl?: () => void;
onOpenCatalogManager: () => void;
onToggleSelectMode: () => void;
onSelectAll: () => void;
@@ -40,6 +41,7 @@ const LibraryHeader: React.FC<LibraryHeaderProps> = ({
onPullLibrary,
onImportBooksFromFiles,
onImportBooksFromDirectory,
onImportBookFromUrl,
onOpenCatalogManager,
onToggleSelectMode,
onSelectAll,
@@ -161,6 +163,7 @@ const LibraryHeader: React.FC<LibraryHeaderProps> = ({
<ImportMenu
onImportBooksFromFiles={onImportBooksFromFiles}
onImportBooksFromDirectory={onImportBooksFromDirectory}
onImportBookFromUrl={onImportBookFromUrl}
onOpenCatalogManager={onOpenCatalogManager}
/>
</Dropdown>
+42
View File
@@ -84,6 +84,10 @@ import GroupHeader from './components/GroupHeader';
import ImportFromFolderDialog, {
ImportFromFolderResult,
} from './components/ImportFromFolderDialog';
import ImportFromUrlDialog from './components/ImportFromUrlDialog';
import { convertToEpubWithWorker } from '@/services/send/conversion/conversionWorker';
import { getClipOptions } from '@/services/send/clipOptions';
import { invoke } from '@tauri-apps/api/core';
import useShortcuts from '@/hooks/useShortcuts';
import { useReplicaPull } from '@/hooks/useReplicaPull';
import { useCustomFonts } from '@/hooks/useCustomFonts';
@@ -163,6 +167,7 @@ const LibraryPageContent = ({ searchParams }: { searchParams: ReadonlyURLSearchP
const [showCatalogManager, setShowCatalogManager] = useState(
searchParams?.get('opds') === 'true',
);
const [showImportFromUrl, setShowImportFromUrl] = useState(false);
const [loading, setLoading] = useState(false);
const [libraryLoaded, setLibraryLoaded] = useState(false);
const [isSelectMode, setIsSelectMode] = useState(false);
@@ -884,6 +889,37 @@ const LibraryPageContent = ({ searchParams }: { searchParams: ReadonlyURLSearchP
});
};
const handleImportBookFromUrl = async (url: string) => {
// Tauri-only. Routes through the Rust `clip_url` command which spawns
// a hidden Tauri webview, loads the URL with the real browser engine
// (correct TLS fingerprint, runs the page's JS, executes any
// Cloudflare challenge), then captures `document.documentElement
// .outerHTML` and returns it. End to end this is exactly the local-
// file path — no inbox, no upload-then-download, no server round-trip
// — `importBooks` is the same call drag-drop uses.
if (!isTauriAppPlatform()) return;
console.log('[clip] start', { url });
setIsSelectMode(false);
const t0 = performance.now();
const html = await invoke<string>('clip_url', { url, options: getClipOptions(_) });
console.log('[clip] fetched', {
bytes: html.length,
ms: Math.round(performance.now() - t0),
});
const t1 = performance.now();
const book = await convertToEpubWithWorker({ kind: 'page', html, url });
console.log('[clip] epub built', {
title: book.title,
author: book.author || undefined,
bytes: book.file.size,
ms: Math.round(performance.now() - t1),
});
const groupId = searchParams?.get('group') || '';
console.log('[clip] importing locally', { name: book.file.name, groupId: groupId || null });
await importBooks([{ file: book.file }], groupId);
console.log('[clip] done');
};
const handleImportBooksFromDirectory = async (dirPath?: string) => {
if (!appService || !isTauriAppPlatform()) return;
@@ -1077,6 +1113,7 @@ const LibraryPageContent = ({ searchParams }: { searchParams: ReadonlyURLSearchP
onImportBooksFromDirectory={
appService?.canReadExternalDir ? handleImportBooksFromDirectory : undefined
}
onImportBookFromUrl={isTauriAppPlatform() ? () => setShowImportFromUrl(true) : undefined}
onOpenCatalogManager={handleShowOPDSDialog}
onToggleSelectMode={() => handleSetSelectMode(!isSelectMode)}
onSelectAll={handleSelectAll}
@@ -1239,6 +1276,11 @@ const LibraryPageContent = ({ searchParams }: { searchParams: ReadonlyURLSearchP
}}
/>
)}
<ImportFromUrlDialog
isOpen={showImportFromUrl}
onClose={() => setShowImportFromUrl(false)}
onSubmit={handleImportBookFromUrl}
/>
<Toast />
</div>
);
+54 -33
View File
@@ -1,19 +1,20 @@
'use client';
import { useCallback, useRef, useState } from 'react';
import { MdUploadFile, MdCheckCircle, MdError, MdLink } from 'react-icons/md';
import { MdUploadFile, MdCheckCircle, MdError, MdLink, MdExtension } from 'react-icons/md';
import { useEnv } from '@/context/EnvContext';
import { useAuth } from '@/context/AuthContext';
import { useSettingsStore } from '@/store/settingsStore';
import { useLibraryStore } from '@/store/libraryStore';
import { useTranslation } from '@/hooks/useTranslation';
import { fetchWithAuth } from '@/utils/fetch';
import { getAPIBaseUrl, isTauriAppPlatform } from '@/services/environment';
import { isTauriAppPlatform } from '@/services/environment';
import { ingestFile } from '@/services/ingestService';
import {
convertFileIfNeeded,
convertToEpubWithWorker,
} from '@/services/send/conversion/conversionWorker';
import { getClipOptions } from '@/services/send/clipOptions';
import { invoke } from '@tauri-apps/api/core';
type ItemStatus = 'working' | 'done' | 'error';
@@ -34,6 +35,12 @@ export default function SendPage() {
const { envConfig, appService } = useEnv();
const { user } = useAuth();
const { settings } = useSettingsStore();
// Direct client fetch only works without CORS, i.e. inside the Tauri
// webview. On the pure web build the URL flow has no reliable way to scrape
// the full article (server-proxied fetches lose to bot detection / login
// walls / JS rendering), so the URL field is hidden and the user is pointed
// at the browser extension instead.
const canClipFromUrl = isTauriAppPlatform();
const [items, setItems] = useState<SendItem[]>([]);
const [dragging, setDragging] = useState(false);
@@ -85,24 +92,24 @@ export default function SendPage() {
const id = crypto.randomUUID();
setItems((prev) => [...prev, { id, label: target, status: 'working' }]);
try {
let html: string;
if (isTauriAppPlatform()) {
const res = await fetch(target);
if (!res.ok) throw new Error(`Could not fetch URL (${res.status})`);
html = await res.text();
} else {
const res = await fetchWithAuth(
`${getAPIBaseUrl()}/send/fetch-url?url=${encodeURIComponent(target)}`,
{ method: 'GET' },
);
html = ((await res.json()) as { html: string }).html;
}
const book = await convertToEpubWithWorker({ kind: 'article', html, url: target });
// Tauri-only: route through the Rust `clip_url` command which
// spawns a hidden webview, lets the real browser engine load the
// URL (so TLS fingerprint + JS challenges resolve naturally) and
// returns `document.documentElement.outerHTML`. On web we never
// reach here — the URL field is hidden.
const html = await invoke<string>('clip_url', { url: target, options: getClipOptions(_) });
const book = await convertToEpubWithWorker({ kind: 'page', html, url: target });
await importResolvedFile(book.file, id, book.title);
} catch (err) {
const detail =
err instanceof Error
? err.message
: typeof err === 'string'
? err
: _('Could not fetch this page');
setItem(id, {
status: 'error',
detail: err instanceof Error ? err.message : _('Could not fetch this page'),
detail,
});
}
}, [url, importResolvedFile, setItem, _]);
@@ -163,22 +170,36 @@ export default function SendPage() {
}}
/>
<div className='flex gap-2'>
<input
type='url'
className='input input-bordered eink-bordered flex-1'
placeholder={_('Paste an article URL')}
value={url}
onChange={(e) => setUrl(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') void handleUrl();
}}
/>
<button type='button' className='btn btn-contrast' onClick={() => void handleUrl()}>
<MdLink className='h-4 w-4' />
{_('Add')}
</button>
</div>
{canClipFromUrl ? (
<div className='flex gap-2'>
<input
type='url'
className='input input-bordered eink-bordered flex-1'
placeholder={_('Paste an article URL')}
value={url}
onChange={(e) => setUrl(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') void handleUrl();
}}
/>
<button type='button' className='btn btn-contrast' onClick={() => void handleUrl()}>
<MdLink className='h-4 w-4' />
{_('Add')}
</button>
</div>
) : (
<section className='card eink-bordered border-base-200 bg-base-100 flex flex-col gap-2 border p-5'>
<div className='flex items-center gap-2'>
<MdExtension className='text-base-content/70 h-5 w-5 flex-shrink-0' />
<h2 className='text-sm font-medium'>{_('Send a web article')}</h2>
</div>
<p className='text-base-content/70 text-xs leading-relaxed'>
{_(
'Install the Readest browser extension to send the article you are reading to your library — it clips the page from your browser so paywalled and login-only sites still work.',
)}
</p>
</section>
)}
{items.length > 0 && (
<ul className='card eink-bordered border-base-200 bg-base-100 divide-base-200 divide-y overflow-hidden border'>
+15 -6
View File
@@ -41,7 +41,6 @@ export function useInboxDrainer(): void {
const { envConfig, appService } = useEnv();
const { user } = useAuth();
const { settings } = useSettingsStore();
const libraryLoaded = useLibraryStore((s) => s.libraryLoaded);
const runningRef = useRef(false);
const lastDrainAtRef = useRef(0);
@@ -98,6 +97,19 @@ export function useInboxDrainer(): void {
const filename = item.filename ?? 'document';
if (item.kind === 'html') {
// A captured page from the bookmarklet / browser extension carries
// `url` (the source); run Readability so the EPUB is just the
// article body, not the surrounding page chrome. A standalone
// .html attachment (no url) gets converted as a document.
if (item.url) {
const html = new TextDecoder('utf-8').decode(bytes);
const book = await convertToEpubWithWorker({
kind: 'article',
html,
url: item.url,
});
return book.file;
}
const book = await convertToEpubWithWorker({
kind: 'html',
bytes,
@@ -163,10 +175,7 @@ export function useInboxDrainer(): void {
}, [user, appService, settings, envConfig]);
useEffect(() => {
// Hold off until the library has loaded — otherwise the very first
// drained item would force `updateBooks` to fall back to its own disk
// load. Once `libraryLoaded` flips true this effect re-runs.
if (!user || !libraryLoaded) return;
if (!user) return;
void runDrain();
const interval = setInterval(() => void runDrain(), DRAIN_INTERVAL_MS);
const onFocus = () => void runDrain();
@@ -175,7 +184,7 @@ export function useInboxDrainer(): void {
clearInterval(interval);
window.removeEventListener('focus', onFocus);
};
}, [user, libraryLoaded, runDrain]);
}, [user, runDrain]);
}
/**
+64 -10
View File
@@ -2,11 +2,13 @@ import type { NextApiRequest, NextApiResponse } from 'next';
import { createSupabaseAdminClient } from '@/utils/supabase';
import { corsAllMethods, runMiddleware } from '@/utils/cors';
import { validateUserAndToken } from '@/utils/access';
import { putObject } from '@/utils/object';
import { parseSubjectTag } from '@/services/send/sendAddress';
import { SEND_INBOX_PENDING_LIMIT } from '@/services/constants';
import { SEND_INBOX_BUCKET, SEND_INBOX_PENDING_LIMIT } from '@/services/constants';
import type { DBSendInboxItem } from '@/types/sendRecords';
const RECENT_LIMIT = 20;
const MAX_CLIP_HTML_BYTES = 5 * 1024 * 1024;
/**
* Inbox endpoint clients route through here instead of querying Supabase
@@ -43,15 +45,9 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
return res.status(405).json({ error: 'Method not allowed' });
}
const url = String(req.body?.url ?? '').trim();
if (!url || !/^https?:\/\//i.test(url)) {
return res.status(400).json({ error: 'A valid http(s) URL is required' });
}
const title = req.body?.title ? String(req.body.title) : null;
// Anti-abuse: cap undrained items so a leaked address/token can't flood R2
// or the user's library. Count claimed items too — a crashed drainer can
// leave items stuck in `claimed` until the lease expires.
// Anti-abuse: cap undrained items so a leaked token can't flood R2 or the
// user's library. Count claimed items too — a crashed drainer can leave
// items stuck in `claimed` until the lease expires.
const { count, error: countError } = await supabase
.from('send_inbox')
.select('id', { count: 'exact', head: true })
@@ -64,6 +60,64 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
return res.status(429).json({ error: 'Inbox is full — open Readest to process pending items' });
}
const kind = String(req.body?.kind ?? 'url');
if (kind === 'html') {
// Bookmarklet / extension path: caller posts the page's rendered HTML.
// This bypasses bot-protection that would defeat a server-side fetch
// (CAPTCHAs, login walls, JS-rendered content). The HTML lands in R2 and
// the drainer converts it identically to the email-attachment flow.
const html = String(req.body?.html ?? '');
if (!html) return res.status(400).json({ error: 'html is required' });
const bytes = new TextEncoder().encode(html);
if (bytes.byteLength > MAX_CLIP_HTML_BYTES) {
return res.status(413).json({ error: 'Page is too large to send' });
}
const title = req.body?.title ? String(req.body.title).slice(0, 500) : null;
const sourceUrl = req.body?.url ? String(req.body.url).slice(0, 2000) : null;
const { data: row, error: rowError } = await supabase
.from('send_inbox')
.insert({
user_id: user.id,
kind: 'html',
source: 'extension',
url: sourceUrl,
filename: title,
byte_size: bytes.byteLength,
subject_tag: parseSubjectTag(title) ?? null,
})
.select('id')
.single<{ id: string }>();
if (rowError) return res.status(500).json({ error: rowError.message });
const payloadKey = `inbox/${user.id}/${row.id}/page.html`;
try {
await putObject(payloadKey, bytes.buffer, 'text/html; charset=utf-8', SEND_INBOX_BUCKET);
} catch (err) {
// Roll back the inbox row so we never leave a `pending` item the
// drainer would only fail on.
await supabase.from('send_inbox').delete().eq('id', row.id);
console.error('Inbox clip upload failed:', err);
return res.status(500).json({ error: 'Could not store page' });
}
const { error: updateError } = await supabase
.from('send_inbox')
.update({ payload_key: payloadKey })
.eq('id', row.id);
if (updateError) return res.status(500).json({ error: updateError.message });
return res.status(200).json({ id: row.id });
}
// kind === 'url' (legacy extension path)
const url = String(req.body?.url ?? '').trim();
if (!url || !/^https?:\/\//i.test(url)) {
return res.status(400).json({ error: 'A valid http(s) URL is required' });
}
const title = req.body?.title ? String(req.body.title) : null;
const { data, error } = await supabase
.from('send_inbox')
.insert({
@@ -0,0 +1,39 @@
/**
* Options handed to the Rust `clip_url` command so the in-webview
* loading overlay, window title, "Saved" page, and native window
* background all match Readest's current theme + UI language.
*
* Each `_()` call is a literal string so the i18next scanner can
* extract the keys keep them inline here rather than building from
* dynamic input. The theme `bg`/`fg` come from the same
* `getThemeCode()` that paints the rest of the app, so a user on
* light, dark, eink, or a custom palette sees the same chrome in the
* clipper window.
*/
import { getThemeCode } from '@/utils/style';
type Translate = (key: string) => string;
export interface ClipOptions {
windowTitle: string;
overlayTitle: string;
loadingStatus: string;
capturingStatus: string;
savedTitle: string;
background: string;
foreground: string;
}
export function getClipOptions(_: Translate): ClipOptions {
const { bg, fg } = getThemeCode();
return {
windowTitle: _('Saving to your Readest library…'),
overlayTitle: _('Saving to Readest'),
loadingStatus: _('Loading article…'),
capturingStatus: _('Capturing article…'),
savedTitle: _('Saved to Readest'),
background: bg,
foreground: fg,
};
}
@@ -0,0 +1,365 @@
import { fetch as tauriFetch } from '@tauri-apps/plugin-http';
import { isTauriAppPlatform } from '@/services/environment';
import { imageFetchHeaders } from './httpHeaders';
import type { EpubImage } from './types';
// On Tauri we go through the Rust HTTP client (no browser-CORS); on web
// we use the native fetch — the bundler is only ever invoked from a
// CORS-free caller there (the future extension's content script). In
// Tauri we also fold in the full image-fetch header set (UA + Sec-Ch-Ua +
// Sec-Fetch-* + Referer) so CDNs that gate images on the browser shape
// — NYT, WSJ, paywalled CDNs — cooperate.
const httpFetch = (url: string, referer: string | null, init?: RequestInit): Promise<Response> => {
if (!isTauriAppPlatform()) return globalThis.fetch(url, init);
const baseHeaders = imageFetchHeaders(referer);
const headers = new Headers(init?.headers);
for (const [k, v] of Object.entries(baseHeaders)) {
if (!headers.has(k)) headers.set(k, v);
}
return tauriFetch(url, { ...init, headers });
};
/** Per-asset limits picked to keep clipped articles light. */
export const MAX_ASSET_BYTES = 5 * 1024 * 1024;
export const MAX_TOTAL_ASSET_BYTES = 30 * 1024 * 1024;
const FETCH_TIMEOUT_MS = 8_000;
const MAX_CONCURRENCY = 4;
const MIME_TO_EXT: Record<string, string> = {
'image/jpeg': 'jpg',
'image/jpg': 'jpg',
'image/png': 'png',
'image/gif': 'gif',
'image/webp': 'webp',
'image/svg+xml': 'svg',
'image/avif': 'avif',
'image/bmp': 'bmp',
};
/** Attributes that aren't useful in the bundled EPUB. Removed AFTER the URL
* has been picked `srcset` and the `data-*` variants can be the source. */
const STRIP_ATTRS = [
'loading',
'decoding',
'fetchpriority',
'crossorigin',
'srcset',
'data-src',
'data-original',
'data-lazy',
'data-actualsrc',
'data-srcset',
];
/**
* Pick the highest-resolution candidate from a `srcset` value. Medium,
* Substack, NYT and other modern publishers set `src` to a ~60px LQIP
* placeholder for lazy-loading and put the real image along with its
* other responsive variants in `srcset` with width (`Nw`) or density
* (`Nx`) descriptors. Picking the largest variant gives the EPUB
* full-quality images, not blurry placeholders.
*/
function pickLargestFromSrcset(srcset: string): string | null {
let bestUrl: string | null = null;
let bestScore = -1;
for (const entry of srcset.split(',')) {
const parts = entry.trim().split(/\s+/);
let url = parts[0] || '';
if (!url) continue;
if (url.startsWith('//')) url = `https:${url}`;
if (!/^https?:/i.test(url) && !url.startsWith('/')) continue;
// No descriptor → score 1 (treat as 1x). Width descriptors (`1280w`)
// and density descriptors (`2x`) sort naturally on the numeric prefix.
let score = 1;
const desc = parts[1];
if (desc) {
const m = desc.match(/^(\d+(?:\.\d+)?)([wx])$/i);
if (m) score = parseFloat(m[1]!);
}
if (score > bestScore) {
bestScore = score;
bestUrl = url;
}
}
return bestUrl;
}
/**
* Resolve an `<img>` to a fetchable URL, preferring high-resolution
* variants in `srcset` over what `src` likely contains (a tiny
* placeholder on lazy-loading sites). Falls back through the common
* `data-*` lazy-load attribute conventions before giving up.
*/
function pickImgUrl(img: Element): string | null {
// Prefer srcset — most likely the real, high-res image.
const srcset = img.getAttribute('srcset');
if (srcset) {
const picked = pickLargestFromSrcset(srcset);
if (picked) return picked;
}
const dataSrcset = img.getAttribute('data-srcset');
if (dataSrcset) {
const picked = pickLargestFromSrcset(dataSrcset);
if (picked) return picked;
}
// Lazy-load `data-*` attributes (older lazy-loaders / non-srcset CMSes).
const lazy =
img.getAttribute('data-src') ||
img.getAttribute('data-original') ||
img.getAttribute('data-lazy') ||
img.getAttribute('data-actualsrc');
if (lazy && lazy.trim()) return lazy;
// Fall back to `src` last — on a lazy-loading site this is the LQIP,
// but on a vanilla site it's the only thing set.
const direct = img.getAttribute('src');
if (direct && direct.trim()) return direct;
return null;
}
/** Resolve a `<picture>`/`<source>` to its highest-resolution image URL.
* Takes the first `<source>` that yields a usable URL `<picture>`
* authors order sources by preference, with the largest variant inside
* each source's srcset. */
function pickPictureUrl(picture: Element): string | null {
for (const source of picture.querySelectorAll('source')) {
const srcset = source.getAttribute('srcset');
if (srcset) {
const picked = pickLargestFromSrcset(srcset);
if (picked) return picked;
}
const src = source.getAttribute('src');
if (src) return src;
}
const fallback = picture.querySelector('img');
return fallback ? pickImgUrl(fallback) : null;
}
function dropElement(el: Element): void {
el.parentNode?.removeChild(el);
}
/** Hex-encode a `Uint8Array`. ~10× faster than `Array.prototype.map(toString(16))`. */
function hex(bytes: Uint8Array, max = 16): string {
const out = new Array<string>(max);
for (let i = 0; i < max && i < bytes.length; i++) {
out[i] = bytes[i]!.toString(16).padStart(2, '0');
}
return out.join('');
}
async function sha256(bytes: ArrayBuffer): Promise<Uint8Array> {
const digest = await crypto.subtle.digest('SHA-256', bytes);
return new Uint8Array(digest);
}
function extFromMime(mime: string, fallbackUrl: string): string {
const base = mime.split(';')[0]!.trim().toLowerCase();
if (MIME_TO_EXT[base]) return MIME_TO_EXT[base]!;
const m = fallbackUrl.match(/\.([a-z0-9]{2,5})(?:\?|#|$)/i);
return m ? m[1]!.toLowerCase() : 'bin';
}
function mimeFromContentType(contentType: string | null, url: string): string {
if (contentType) {
const base = contentType.split(';')[0]!.trim().toLowerCase();
if (base.startsWith('image/')) return base;
}
// Fall back to the URL extension — some CDNs return `application/octet-stream`.
const m = url.match(/\.(jpe?g|png|gif|webp|svg|avif|bmp)(?:\?|#|$)/i);
if (m) {
const e = m[1]!.toLowerCase();
if (e === 'jpg' || e === 'jpeg') return 'image/jpeg';
if (e === 'svg') return 'image/svg+xml';
return `image/${e}`;
}
return 'application/octet-stream';
}
interface FetchedAsset {
url: string;
path: string;
bytes: ArrayBuffer;
mime: string;
}
async function fetchAsset(url: string, referer: string | null): Promise<FetchedAsset | null> {
const ac = new AbortController();
const timer = setTimeout(() => ac.abort(), FETCH_TIMEOUT_MS);
try {
const res = await httpFetch(url, referer, { signal: ac.signal, redirect: 'follow' });
if (!res.ok) return null;
const bytes = await res.arrayBuffer();
if (bytes.byteLength === 0) return null;
if (bytes.byteLength > MAX_ASSET_BYTES) return null;
const mime = mimeFromContentType(res.headers.get('content-type'), url);
if (!mime.startsWith('image/')) return null;
const digest = await sha256(bytes);
const ext = extFromMime(mime, url);
const path = `images/${hex(digest, 16)}.${ext}`;
return { url, path, bytes, mime };
} finally {
clearTimeout(timer);
}
}
export interface BundleAssetsResult {
/** Rewritten HTML with `<img src>` pointing at the inlined paths. */
html: string;
/** Image resources to embed in the EPUB. */
images: EpubImage[];
/** Number of references that could not be fetched (404, timeout, oversize). */
missing: number;
}
/**
* Walk an extracted-article HTML fragment, fetch every referenced image under
* the page's own session (cookies, real UA only safe to call from a context
* with CORS bypassed, i.e. Tauri webview or browser-extension content script),
* and return a rewritten HTML fragment plus the inlined image resources.
*
* Honours lazy-load attribute conventions (`data-src`, `data-original`,
* `data-srcset`, `srcset`). Drops `<iframe>`/`<video>`/`<audio>`/`<embed>`/
* `<object>` entirely. Keeps inline `<svg>` as-is. A failed fetch becomes an
* alt-text placeholder so the EPUB still builds.
*/
export async function bundleAssets(
contentHtml: string,
pageUrl: string,
): Promise<BundleAssetsResult> {
const doc = new DOMParser().parseFromString(`<div id="root">${contentHtml}</div>`, 'text/html');
const root = doc.getElementById('root');
if (!root) return { html: contentHtml, images: [], missing: 0 };
// Drop unrenderable / blocking media before walking.
for (const el of Array.from(root.querySelectorAll('iframe, video, audio, embed, object'))) {
dropElement(el);
}
// Pick every fetchable image URL.
const targets: { el: Element; url: string }[] = [];
for (const picture of Array.from(root.querySelectorAll('picture'))) {
const url = pickPictureUrl(picture);
if (!url) {
dropElement(picture);
continue;
}
// Flatten <picture> to a plain <img> so the rewritten HTML resolves
// against the in-EPUB path without needing <source> entries the EPUB
// reader probably doesn't honour.
const img = doc.createElement('img');
const alt = picture.querySelector('img')?.getAttribute('alt') ?? '';
if (alt) img.setAttribute('alt', alt);
picture.replaceWith(img);
let abs: string;
try {
abs = new URL(url, pageUrl).toString();
} catch {
dropElement(img);
continue;
}
targets.push({ el: img, url: abs });
}
for (const img of Array.from(root.querySelectorAll('img'))) {
if (targets.some((t) => t.el === img)) {
// Already queued via <picture>. Just clean up noise attrs.
for (const attr of STRIP_ATTRS) img.removeAttribute(attr);
continue;
}
const url = pickImgUrl(img);
// Strip noise attrs AFTER picking — srcset / data-* may be the source.
for (const attr of STRIP_ATTRS) img.removeAttribute(attr);
if (!url) {
dropElement(img);
continue;
}
let abs: string;
try {
abs = new URL(url, pageUrl).toString();
} catch {
dropElement(img);
continue;
}
// Tiny tracking pixels: cheap to spot ahead of the fetch.
if (img.getAttribute('width') === '1' && img.getAttribute('height') === '1') {
dropElement(img);
continue;
}
targets.push({ el: img, url: abs });
}
// Dedupe by URL so a hero shared by `<picture>` and `<img>` only fetches once.
const urlToTargets = new Map<string, Element[]>();
for (const { el, url } of targets) {
const list = urlToTargets.get(url) ?? [];
list.push(el);
urlToTargets.set(url, list);
}
const uniqueUrls = Array.from(urlToTargets.keys());
console.log('[clip/bundle] start', {
unique_urls: uniqueUrls.length,
sample: uniqueUrls.slice(0, 5),
});
const fetched = new Map<string, FetchedAsset>();
let totalBytes = 0;
let missing = 0;
// Bounded-concurrency fetch loop. `MAX_CONCURRENCY` workers pull from a
// shared cursor so we never hammer a single origin with N requests. Each
// fetch owns its own timeout — see `fetchAsset`.
let cursor = 0;
const worker = async () => {
while (cursor < uniqueUrls.length) {
const i = cursor++;
const url = uniqueUrls[i]!;
if (totalBytes >= MAX_TOTAL_ASSET_BYTES) {
missing++;
continue;
}
try {
const asset = await fetchAsset(url, pageUrl);
if (asset && totalBytes + asset.bytes.byteLength <= MAX_TOTAL_ASSET_BYTES) {
fetched.set(url, asset);
totalBytes += asset.bytes.byteLength;
} else {
missing++;
}
} catch (err) {
missing++;
console.warn('[clip/bundle] image fetch failed', {
url,
error: err instanceof Error ? err.message : String(err),
});
}
}
};
await Promise.all(Array.from({ length: MAX_CONCURRENCY }, worker));
console.log('[clip/bundle] done', {
fetched: fetched.size,
missing,
total_bytes: totalBytes,
});
// Rewrite refs. Failed-fetch elements get their `src` cleared so they
// render as the alt text placeholder.
const images: EpubImage[] = [];
const seenPaths = new Set<string>();
for (const [url, els] of urlToTargets) {
const asset = fetched.get(url);
if (!asset) {
for (const el of els) {
el.removeAttribute('src');
}
continue;
}
if (!seenPaths.has(asset.path)) {
images.push({ path: asset.path, bytes: asset.bytes, mime: asset.mime });
seenPaths.add(asset.path);
}
for (const el of els) {
el.setAttribute('src', asset.path);
}
}
return { html: root.innerHTML, images, missing };
}
@@ -1,5 +1,6 @@
import { configureZip } from '@/utils/zip';
import type { EpubChapter, EpubBuildMetadata } from './types';
import { buildNavMap } from './toc';
import type { EpubChapter, EpubBuildMetadata, EpubImage } from './types';
// Zero the zip timestamps so converting the same source twice yields
// byte-identical EPUBs — that keeps the import-time hash stable and dedups
@@ -19,11 +20,20 @@ const escapeXml = (str: string): string => {
.replace(/'/g, '&apos;');
};
// System font stack only — the bundled EPUB stays self-contained and never
// reaches out to the network when opened offline.
const CSS = `
body { line-height: 1.6; font-size: 1em; text-align: justify; }
body { line-height: 1.6; font-size: 1em; text-align: justify;
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto,
Oxygen-Sans, Ubuntu, Cantarell, "Helvetica Neue", sans-serif; }
h1, h2, h3 { line-height: 1.3; }
p { margin: 0.6em 0; }
img { max-width: 100%; height: auto; }
figure { margin: 1em 0; }
figcaption { font-size: 0.9em; color: #666; text-align: center; }
blockquote { margin: 1em 1.5em; color: #444; border-left: 3px solid #ccc; padding-left: 1em; }
pre, code { font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace; }
pre { white-space: pre-wrap; background: #f5f5f5; padding: 0.6em 0.8em; border-radius: 4px; }
`;
/**
@@ -35,12 +45,13 @@ img { max-width: 100%; height: auto; }
export async function buildEpub(
chapters: EpubChapter[],
metadata: EpubBuildMetadata,
images: EpubImage[] = [],
): Promise<Blob> {
if (chapters.length === 0) {
throw new Error('buildEpub: no chapters');
}
await configureZip();
const { BlobWriter, TextReader, ZipWriter } = await import('@zip.js/zip.js');
const { BlobWriter, TextReader, Uint8ArrayReader, ZipWriter } = await import('@zip.js/zip.js');
const { title, author, language, identifier } = metadata;
const zipWriter = new ZipWriter(new BlobWriter('application/epub+zip'), {
@@ -56,16 +67,24 @@ export async function buildEpub(
</container>`;
await zipWriter.add('META-INF/container.xml', new TextReader(containerXml), zipWriteOptions);
const navPoints = chapters
.map((chapter, i) => {
const id = `chapter${i + 1}`;
return (
`<navPoint id="navPoint-${id}" playOrder="${i + 1}">` +
`<navLabel><text>${escapeXml(chapter.title)}</text></navLabel>` +
`<content src="./OEBPS/${id}.xhtml"/></navPoint>`
);
})
.join('\n');
// Prefer a heading-based nested navMap when the caller supplied one —
// the EPUB reader's TOC sidebar then mirrors the article's section
// structure rather than showing a single "Article Title" entry. Falls
// back to one navPoint per chapter when no headings were extracted
// (a Substack-style image-and-paragraph post with no h2s, etc).
const navPoints =
metadata.toc && metadata.toc.length > 0
? buildNavMap(metadata.toc, 'OEBPS/chapter1.xhtml', escapeXml)
: chapters
.map((chapter, i) => {
const id = `chapter${i + 1}`;
return (
`<navPoint id="navPoint-${id}" playOrder="${i + 1}">` +
`<navLabel><text>${escapeXml(chapter.title)}</text></navLabel>` +
`<content src="./OEBPS/${id}.xhtml"/></navPoint>`
);
})
.join('\n');
const tocNcx = `<?xml version="1.0" encoding="UTF-8"?>
<ncx xmlns="http://www.daisy.org/z3986/2005/ncx/" version="2005-1">
@@ -83,6 +102,16 @@ export async function buildEpub(
await zipWriter.add('style.css', new TextReader(CSS), zipWriteOptions);
// Inline images: keep relative to OEBPS/ so `<img src="images/abc.png">`
// inside a chapter resolves correctly.
for (const image of images) {
await zipWriter.add(
`OEBPS/${image.path}`,
new Uint8ArrayReader(new Uint8Array(image.bytes)),
zipWriteOptions,
);
}
for (let i = 0; i < chapters.length; i++) {
const chapter = chapters[i]!;
const xhtml = `<?xml version="1.0" encoding="UTF-8"?>
@@ -103,6 +132,12 @@ export async function buildEpub(
`<item id="chap${i + 1}" href="OEBPS/chapter${i + 1}.xhtml" media-type="application/xhtml+xml"/>`,
)
.join('\n ');
const imageManifest = images
.map(
(image, i) =>
`<item id="img${i + 1}" href="OEBPS/${escapeXml(image.path)}" media-type="${escapeXml(image.mime)}"/>`,
)
.join('\n ');
const spine = chapters.map((_, i) => `<itemref idref="chap${i + 1}"/>`).join('\n ');
const contentOpf = `<?xml version="1.0" encoding="UTF-8"?>
@@ -115,6 +150,7 @@ export async function buildEpub(
</metadata>
<manifest>
${manifest}
${imageManifest}
<item id="ncx" href="toc.ncx" media-type="application/x-dtbncx+xml"/>
<item id="css" href="style.css" media-type="text/css"/>
</manifest>
@@ -1,4 +1,4 @@
import type { ConvertInput } from './convertToEpub';
import { convertToEpub, type ConvertInput } from './convertToEpub';
import type { ConvertedBook } from './types';
import type {
ConversionWorkerRequest,
@@ -60,32 +60,27 @@ async function convertInWorker(input: ConvertInput, timeoutMs: number): Promise<
});
}
/**
* Main-thread fallback. Dynamic-imported so the heavy conversion deps
* (mammoth, Readability, DOMPurify, zip.js) never load on the main thread
* unless the Worker path is actually unavailable.
*/
async function convertOnMainThread(input: ConvertInput): Promise<ConvertedBook> {
const { convertToEpub } = await import('./convertToEpub');
return convertToEpub(input);
}
/**
* Convert a document to EPUB off the main thread, falling back to in-thread
* conversion when Web Workers are unavailable or the worker fails.
*
* `kind: 'page'` is the exception its asset bundler uses the
* `@tauri-apps/plugin-http` fetch to bypass CORS, and that binding lives on
* `window.__TAURI_INTERNALS__` which a Web Worker can't reach. So page
* clips always run on the main thread.
*/
export async function convertToEpubWithWorker(
input: ConvertInput,
timeoutMs: number = DEFAULT_TIMEOUT_MS,
): Promise<ConvertedBook> {
if (typeof Worker === 'undefined') {
return convertOnMainThread(input);
if (input.kind === 'page' || typeof Worker === 'undefined') {
return convertToEpub(input);
}
try {
return await convertInWorker(input, timeoutMs);
} catch (error) {
console.warn('Conversion worker failed, falling back to main thread:', error);
return convertOnMainThread(input);
return convertToEpub(input);
}
}
@@ -4,8 +4,11 @@ import { TxtToEpubConverter } from '@/utils/txt';
import { detectLanguage } from '@/utils/lang';
import { sanitizeHtml, sanitizeForParsing } from './sanitizeHtml';
import { buildEpub } from './buildEpub';
import { bundleAssets } from './assetBundler';
import { findSiteRule, type SiteRule } from './siteRules';
import { extractHeadings } from './toc';
import { ConversionError } from './types';
import type { ConvertibleMime, ConvertedBook, EpubChapter } from './types';
import type { ConvertibleMime, ConvertedBook, EpubChapter, EpubImage, TocEntry } from './types';
/** Discriminated input — the caller resolves the kind from the MIME type. */
export type ConvertInput =
@@ -13,7 +16,11 @@ export type ConvertInput =
| { kind: 'rtf'; bytes: ArrayBuffer; fileName?: string }
| { kind: 'html'; bytes: ArrayBuffer; fileName?: string }
| { kind: 'txt'; file: File }
| { kind: 'article'; html: string; url: string };
| { kind: 'article'; html: string; url: string }
/** Self-contained page clip: like `article` but fetches every referenced
* image and embeds it in the EPUB. Only safe from a CORS-free context
* (Tauri webview, browser-extension content script). */
| { kind: 'page'; html: string; url: string };
const MIME_TO_KIND: Record<ConvertibleMime, ConvertInput['kind']> = {
'application/vnd.openxmlformats-officedocument.wordprocessingml.document': 'docx',
@@ -57,27 +64,159 @@ function safeFileName(title: string): string {
return base.slice(0, 120);
}
/** Assemble an EPUB from a single block of sanitized HTML. */
async function htmlToBook(rawHtml: string, title: string, author: string): Promise<ConvertedBook> {
const html = sanitizeHtml(rawHtml);
/** Escape text content for inline insertion into HTML. */
function escapeHtml(s: string): string {
return s
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;');
}
/**
* Prepend the article title (as `<h1>`) and the byline (as `<p>`) to the
* extracted body. Readability and the per-site rules give us title and
* byline as separate fields they're stripped out of the body content
* during extraction, so the EPUB chapter would otherwise open with the
* first body paragraph and the user wouldn't see who wrote it or what
* it's called inside the book. The `<h1>` also becomes the top-level
* navMap entry (via `extractHeadings`) so the TOC sidebar shows the
* article title.
*/
function composeArticleContent(title: string, byline: string, body: string): string {
const parts = [`<h1>${escapeHtml(title)}</h1>`];
const trimmedByline = byline.trim();
if (trimmedByline) {
parts.push(`<p>${escapeHtml(trimmedByline)}</p>`);
}
parts.push(body);
return parts.join('\n');
}
/** Assemble an EPUB from a single block of sanitized HTML (optionally with
* pre-fetched image resources to embed). Extracts an in-chapter heading
* TOC so the EPUB reader's sidebar shows the article's sections. */
async function htmlToBook(
rawHtml: string,
title: string,
author: string,
images: EpubImage[] = [],
): Promise<ConvertedBook> {
// Assign stable ids to headings BEFORE sanitization so the strict
// `sanitizeHtml` step (which now allows `id`) preserves them — those
// ids are what `toc.ncx` links to.
const { html: withIds, headings } = extractHeadings(rawHtml);
const html = sanitizeHtml(withIds);
const text = stripTags(html);
if (!text) {
throw new ConversionError('Document has no readable content', 'empty_input');
}
const language = detectLanguage(text.slice(0, 2048)) || 'en';
const chapter: EpubChapter = { title, html };
const blob = await buildEpub([chapter], {
title,
author,
language,
identifier: stableIdentifier(html),
});
const toc: TocEntry[] | undefined = headings.length > 0 ? headings : undefined;
if (toc) {
console.log('[clip/toc] headings', {
count: toc.length,
levels: toc.map((h) => h.level),
});
}
const blob = await buildEpub(
[chapter],
{
title,
author,
language,
identifier: stableIdentifier(html),
toc,
},
images,
);
const file = new File([blob], `${safeFileName(title)}.epub`, {
type: 'application/epub+zip',
});
return { file, title, author };
}
interface RuleExtracted {
content: string;
title?: string;
byline?: string;
}
/**
* Apply a `SiteRule` to a full HTML document. Returns the article body
* (innerHTML of the rule's `content` selector, with `strip` selectors
* removed) plus the rule-extracted title and byline if present.
*
* Returns null when the content selector matches nothing the caller falls
* through to Readability so a stale rule never blocks extraction.
*/
function extractWithSiteRule(rawHtml: string, rule: SiteRule): RuleExtracted | null {
const doc = new DOMParser().parseFromString(sanitizeForParsing(rawHtml), 'text/html');
const contentEl = doc.querySelector(rule.content);
if (!contentEl) return null;
if (rule.strip?.length) {
for (const sel of rule.strip) {
for (const el of Array.from(contentEl.querySelectorAll(sel))) {
el.parentNode?.removeChild(el);
}
}
}
const ruleTitle = rule.title ? (doc.querySelector(rule.title)?.textContent?.trim() ?? '') : '';
const ruleByline = rule.byline ? (doc.querySelector(rule.byline)?.textContent?.trim() ?? '') : '';
return {
content: contentEl.innerHTML,
title: ruleTitle || undefined,
byline: ruleByline || undefined,
};
}
/**
* Pick the article body without Readability useful when its scoring trips
* on a custom layout (e.g. an article body wrapped in a non-semantic div
* that's revealed by JS, plus heavy page chrome that outscores it).
*
* Walks a prioritized selector list, picks the element with the most stripped
* text, returns its innerHTML if it clears the quality floor.
*/
const ARTICLE_FALLBACK_SELECTORS = [
'#js_content',
'article',
'main article',
'main',
'[role="article"]',
'[itemprop="articleBody"]',
'.post-content',
'.entry-content',
'.article-content',
'.post-body',
'.markdown-body',
'#content',
];
function extractArticleFallback(rawHtml: string, minTextChars: number): string | null {
const doc = new DOMParser().parseFromString(sanitizeForParsing(rawHtml), 'text/html');
let bestHtml = '';
let bestText = 0;
for (const selector of ARTICLE_FALLBACK_SELECTORS) {
let els: NodeListOf<Element>;
try {
els = doc.querySelectorAll(selector);
} catch {
continue;
}
for (const el of Array.from(els)) {
const text = stripTags(el.innerHTML);
if (text.length > bestText) {
bestText = text.length;
bestHtml = el.innerHTML;
}
}
}
return bestText >= minTextChars ? bestHtml : null;
}
/** Pull `<title>` / first `<h1>` out of a full HTML document. */
function extractHtmlTitle(html: string, fallback: string): string {
try {
@@ -161,7 +300,99 @@ export async function convertToEpub(input: ConvertInput): Promise<ConvertedBook>
throw new ConversionError('No readable article found at the URL', 'parse_failed');
}
const title = parsed.title?.trim() || extractHtmlTitle(input.html, input.url);
return htmlToBook(parsed.content, title, parsed.byline?.trim() || '');
const byline = parsed.byline?.trim() || '';
return htmlToBook(composeArticleContent(title, byline, parsed.content), title, byline);
}
case 'page': {
// Same as `article`, but fetch every referenced image and inline it so
// the EPUB is fully offline-readable. Only valid from a CORS-free
// caller (Tauri webview or extension content script).
console.log('[clip/page] input', {
url: input.url,
html_bytes: input.html.length,
});
// Fast path: per-site rule. Skips Readability entirely on sites we
// know it mis-scores. Falls through if the rule's content selector
// matches nothing or the extracted text is below the floor.
const rule = findSiteRule(input.url);
if (rule) {
const ruleResult = extractWithSiteRule(input.html, rule);
const ruleText = ruleResult ? stripTags(ruleResult.content) : '';
console.log('[clip/page] site rule', {
name: rule.name,
matched: !!ruleResult,
text_chars: ruleText.length,
title: ruleResult?.title || null,
});
if (ruleResult && ruleText.length >= 400) {
const title = ruleResult.title || extractHtmlTitle(input.html, input.url);
const byline = ruleResult.byline || '';
const bundle = await bundleAssets(ruleResult.content, input.url);
return htmlToBook(
composeArticleContent(title, byline, bundle.html),
title,
byline,
bundle.images,
);
}
}
let parsed: { title?: string | null; content?: string | null; byline?: string | null } | null;
try {
const doc = new DOMParser().parseFromString(sanitizeForParsing(input.html), 'text/html');
parsed = new Readability(doc).parse();
} catch (err) {
console.warn('[clip/page] readability threw', {
error: err instanceof Error ? err.message : String(err),
});
throw new ConversionError(`Could not extract article: ${String(err)}`, 'parse_failed');
}
if (!parsed?.content) {
console.warn('[clip/page] readability returned no content');
throw new ConversionError('No readable article found at the URL', 'parse_failed');
}
let articleHtml = parsed.content;
let contentText = stripTags(articleHtml);
console.log('[clip/page] readability', {
title: parsed.title || null,
byline: parsed.byline || null,
content_chars: articleHtml.length,
text_chars: contentText.length,
});
// Readability sometimes scores the wrong container on sites with custom
// markup (e.g. an article body wrapped in a non-semantic div revealed
// by JS). When the extracted text is tiny but the page HTML is
// substantial, try a known-selector fallback before giving up.
const QUALITY_FLOOR = 400;
if (contentText.length < QUALITY_FLOOR && input.html.length > 50_000) {
const fallback = extractArticleFallback(input.html, QUALITY_FLOOR);
if (fallback) {
articleHtml = fallback;
contentText = stripTags(articleHtml);
console.log('[clip/page] fallback selector picked up the article', {
text_chars: contentText.length,
});
}
}
// Bot-detection screens, paywall stubs and "please enable JavaScript"
// pages all slip past Readability AND the selector fallback with a
// tiny scrap of text. Refuse to import them — never save a junk EPUB.
if (contentText.length < QUALITY_FLOOR) {
throw new ConversionError(
'Could not read this page — it looks like a verification screen or a login wall. Open it in a browser first, or wait for the upcoming browser extension.',
'parse_failed',
);
}
const title = parsed.title?.trim() || extractHtmlTitle(input.html, input.url);
const byline = parsed.byline?.trim() || '';
const bundle = await bundleAssets(articleHtml, input.url);
return htmlToBook(
composeArticleContent(title, byline, bundle.html),
title,
byline,
bundle.images,
);
}
default: {
throw new ConversionError(
@@ -0,0 +1,64 @@
/**
* Browser-mimicking request headers for the URL-clip pipeline. The Tauri
* Rust HTTP client (`@tauri-apps/plugin-http`) sends none of these by
* default; bot-detection on Cloudflare, Medium, NYT and similar gates the
* 403/429 on their *absence* as much as on a non-browser UA. Sending the
* full Chrome set gets us past UA + header-fingerprint sniffing.
*
* Doesn't help against true session/JS challenges (verification screens,
* paywalled members-only reads). Those need the browser extension.
*
* Shared with the future Phase 4 extension's service worker it can
* import the same constants and apply them on its own outbound fetches.
*/
export const BROWSER_UA =
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 ' +
'(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36';
const SEC_CH_UA = '"Chromium";v="124", "Not-A.Brand";v="99", "Google Chrome";v="124"';
/** Headers a real Chrome sends when navigating to a top-level URL. */
export function pageNavigateHeaders(): Record<string, string> {
return {
'user-agent': BROWSER_UA,
accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
'accept-language': 'en-US,en;q=0.9',
'sec-ch-ua': SEC_CH_UA,
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"macOS"',
'sec-fetch-dest': 'document',
'sec-fetch-mode': 'navigate',
'sec-fetch-site': 'none',
'sec-fetch-user': '?1',
'upgrade-insecure-requests': '1',
};
}
/** Headers a real Chrome sends when loading an image from a page. */
export function imageFetchHeaders(referer: string | null): Record<string, string> {
const headers: Record<string, string> = {
'user-agent': BROWSER_UA,
accept: 'image/avif,image/webp,image/png,image/svg+xml,image/*,*/*;q=0.8',
'accept-language': 'en-US,en;q=0.9',
'sec-ch-ua': SEC_CH_UA,
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"macOS"',
'sec-fetch-dest': 'image',
'sec-fetch-mode': 'no-cors',
'sec-fetch-site': 'cross-site',
};
// Some CDNs gate image responses on a same-origin or page-origin Referer
// (NYT, WSJ, many news image CDNs do this).
if (referer) headers['referer'] = referer;
return headers;
}
/**
* True if the HTTP status code is the shape an anti-bot CDN typically
* returns to a non-browser request used to upgrade the user-facing error
* to something explanatory instead of a bare status code.
*/
export function isLikelyBotBlock(status: number): boolean {
return status === 401 || status === 403 || status === 429 || status === 503;
}
@@ -50,7 +50,10 @@ export function sanitizeHtml(html: string): string {
'figure',
'figcaption',
],
ALLOWED_ATTR: ['href', 'src', 'alt', 'title', 'colspan', 'rowspan'],
// `id` is allowed so heading anchors survive — the EPUB's nested
// navMap uses `chapter1.xhtml#heading-id` to link the TOC sidebar to
// each section. Without it readers see one entry per chapter only.
ALLOWED_ATTR: ['href', 'src', 'alt', 'title', 'colspan', 'rowspan', 'id'],
// Drop anything that would load or run remote code.
FORBID_TAGS: ['script', 'style', 'iframe', 'object', 'embed', 'form', 'input'],
FORBID_ATTR: ['srcset'],
@@ -0,0 +1,69 @@
/**
* Per-site extraction rules. Run *before* Readability when a rule matches,
* we trust the selectors over Readability's scoring, because sites that fail
* Readability fail it consistently (custom containers, JS-revealed content,
* marketing chrome that scores higher than the article).
*
* Stays a fast path, not a wall: if a rule's content selector doesn't match
* or yields too little text, the pipeline falls through to Readability and
* then to the generic selector fallback. A stale rule never silently loses
* content worst case it adds one no-op query before the real extraction.
*
* Designed as a pure-data module with no DOM dependencies so the future
* Phase 4 browser extension's content script can import it verbatim and run
* the same rules in the user's authenticated tab.
*/
export interface SiteRule {
name: string;
/** Match by parsed URL. Hostname is the most reliable; URL pathnames
* change too often. */
match: (url: URL) => boolean;
/** CSS selector for the article body. The element's innerHTML becomes
* the chapter content. */
content: string;
/** Override Readability's title. */
title?: string;
/** Override Readability's byline. */
byline?: string;
/** Selectors to remove from the content before bundling (e.g. "open in
* app" CTAs, QR codes, share buttons). */
strip?: string[];
}
const RULES: SiteRule[] = [
{
// Articles wrap the body in `<div id="js_content">` with
// `visibility:hidden; opacity:0`, then JS reveals it. Readability mis-
// scores the page because the recommendation rail outweighs the
// hidden article in its heuristics.
name: 'mp.weixin.qq.com',
match: (u) => u.hostname === 'mp.weixin.qq.com',
content: '#js_content',
title: '#activity-name, h1.rich_media_title',
byline: '#js_name, .rich_media_meta_nickname',
strip: [
// Inline QR codes (PC + mobile variants)
'.qr_code_pc',
'#js_pc_qr_code',
// Reward / tip popups
'.reward_area',
'.reward_qrcode_area',
'.reward_user_info',
// Promotional inserts
'.promotion_card',
// Cross-app CTAs
'.weui-dialog_appmsg',
'.open_in_weixin_wrap',
],
},
];
/** Find the rule matching `url`, if any. Returns null on parse error. */
export function findSiteRule(url: string): SiteRule | null {
try {
const parsed = new URL(url);
return RULES.find((r) => r.match(parsed)) ?? null;
} catch {
return null;
}
}
@@ -0,0 +1,108 @@
import type { TocEntry } from './types';
/**
* Walk an article HTML fragment, find every `<h1>``<h6>`, assign each a
* stable `id` if missing, and return:
* - the rewritten HTML (headings now carry IDs)
* - the heading list in document order
*
* Used as a pre-step before sanitizing for the EPUB chapter so the
* fragment IDs referenced by `toc.ncx` actually exist in the chapter
* XHTML. Stable slugs mean a re-convert of the same article produces a
* byte-identical EPUB (combined with the zeroed zip timestamps), which
* keeps the import-time hash stable.
*/
export function extractHeadings(html: string): { html: string; headings: TocEntry[] } {
const doc = new DOMParser().parseFromString(
'<!doctype html><html><body></body></html>',
'text/html',
);
const root = doc.createElement('div');
root.id = '__toc_root';
root.innerHTML = html;
doc.body.appendChild(root);
if (!root) return { html, headings: [] };
const headings: TocEntry[] = [];
const used = new Set<string>();
let fallbackCounter = 0;
for (const el of Array.from(root.querySelectorAll('h1, h2, h3, h4, h5, h6'))) {
const text = (el.textContent || '').replace(/\s+/g, ' ').trim();
if (!text) continue;
// Honour the page's existing id if it looks usable; otherwise slug
// from the heading text.
let id = (el.getAttribute('id') || '').trim();
if (!id) {
id = slugify(text);
}
if (!id) {
fallbackCounter++;
id = `h-${fallbackCounter}`;
}
// De-duplicate. Two `## Intro` sections become `intro` + `intro-2`.
let unique = id;
let n = 2;
while (used.has(unique)) {
unique = `${id}-${n}`;
n++;
}
used.add(unique);
el.setAttribute('id', unique);
const level = parseInt(el.tagName.charAt(1), 10);
headings.push({ id: unique, text, level });
}
return { html: root.innerHTML, headings };
}
/** Slug a heading title into a fragment-id-safe string. CJK / emoji /
* other non-Latin scripts strip entirely the caller falls back to
* `h-{n}` so we never emit an empty id. */
function slugify(text: string): string {
return text
.toLowerCase()
.replace(/[\s_]+/g, '-')
.replace(/[^a-z0-9-]/g, '')
.replace(/-+/g, '-')
.replace(/^-|-$/g, '')
.slice(0, 60);
}
/**
* Render the nested `<navPoint>` tree for `toc.ncx` `<navMap>`. Each
* heading becomes a navPoint nested under any prior heading of a
* shallower level (`h2` under `h1`, `h3` under `h2`, etc.). `playOrder`
* is a flat global counter EPUB 2 requires it to be unique and start
* at 1.
*/
export function buildNavMap(
toc: TocEntry[],
chapterFile: string,
escapeXml: (s: string) => string,
): string {
if (toc.length === 0) return '';
const parts: string[] = [];
const stack: number[] = [];
let order = 0;
for (const h of toc) {
while (stack.length && stack[stack.length - 1]! >= h.level) {
stack.pop();
parts.push('</navPoint>');
}
order++;
parts.push(
`<navPoint id="np${order}" playOrder="${order}">` +
`<navLabel><text>${escapeXml(h.text)}</text></navLabel>` +
`<content src="${escapeXml(chapterFile)}#${escapeXml(h.id)}"/>`,
);
stack.push(h.level);
}
while (stack.length) {
stack.pop();
parts.push('</navPoint>');
}
return parts.join('\n');
}
@@ -21,6 +21,14 @@ export interface EpubChapter {
html: string;
}
/** An image resource embedded directly in the EPUB. Path is relative to
* `OEBPS/` and must match the rewritten `<img src>` in the chapter HTML. */
export interface EpubImage {
path: string;
bytes: ArrayBuffer;
mime: string;
}
export interface EpubBuildMetadata {
title: string;
author: string;
@@ -28,6 +36,20 @@ export interface EpubBuildMetadata {
/** Stable EPUB `dc:identifier` derive deterministically so re-converting
* the same source yields the same bytes and dedups on import. */
identifier: string;
/** Optional in-chapter heading TOC. When present, `toc.ncx` `navMap`
* uses these instead of a single chapter-level navPoint the EPUB
* reader's sidebar then shows the article's section structure. */
toc?: TocEntry[];
}
/** One entry in the EPUB's nested TOC a single `<h1>``<h6>` from the
* article. `level` is 16; `chapterIndex` is 0 for now (page clips are
* single-chapter). */
export interface TocEntry {
id: string;
text: string;
level: number;
chapterIndex?: number;
}
export class ConversionError extends Error {
+16
View File
@@ -33,6 +33,22 @@ export const getUploadSignedUrl = async (
}
};
export const putObject = async (
fileKey: string,
body: ArrayBuffer | string,
contentType: string,
bucketName?: string,
) => {
const storageType = getStorageType();
if (storageType === 'r2') {
bucketName = bucketName || process.env['R2_BUCKET_NAME'] || '';
return await r2Storage.putObject(bucketName, fileKey, body, contentType);
} else {
bucketName = bucketName || process.env['S3_BUCKET_NAME'] || '';
return await s3Storage.putObject(bucketName, fileKey, body, contentType);
}
};
export const deleteObject = async (fileKey: string, bucketName?: string) => {
const storageType = getStorageType();
if (storageType === 'r2') {
+13
View File
@@ -54,6 +54,19 @@ export const r2Storage = {
).url.toString();
},
putObject: async (
bucketName: string,
fileKey: string,
body: ArrayBuffer | string,
contentType: string,
) => {
return await r2Storage.getR2Client().fetch(`${r2Storage.getR2Url()}/${bucketName}/${fileKey}`, {
method: 'PUT',
headers: { 'Content-Type': contentType },
body,
});
},
deleteObject: async (bucketName: string, fileKey: string) => {
return await r2Storage.getR2Client().fetch(`${r2Storage.getR2Url()}/${bucketName}/${fileKey}`, {
method: 'DELETE',
+15
View File
@@ -69,6 +69,21 @@ export const s3Storage = {
return uploadUrl;
},
putObject: async (
bucketName: string,
fileKey: string,
body: ArrayBuffer | string,
contentType: string,
) => {
const putCommand = new PutObjectCommand({
Bucket: bucketName,
Key: fileKey,
Body: body instanceof ArrayBuffer ? new Uint8Array(body) : body,
ContentType: contentType,
});
return await s3Storage.getClient().send(putCommand);
},
deleteObject: async (bucketName: string, fileKey: string) => {
const deleteCommand = new DeleteObjectCommand({
Bucket: bucketName,