Update OpenWiki for source ingest
All checks were successful
CI / build-and-deploy (push) Successful in 16s
All checks were successful
CI / build-and-deploy (push) Successful in 16s
This commit is contained in:
Krishna Kumar
@@ -7,7 +7,7 @@ This page explains how the major Marks runtime pieces fit together. For product-
|
||||
`/project.yml` is the XcodeGen project definition. It defines four targets:
|
||||
|
||||
- `Marks` — the main iOS app. Sources include `/Marks` plus `/MarksWidget/WidgetDataStore.swift` so the app can write widget data.
|
||||
- `ShareExtension` — app extension with `/ShareExtension` plus selected shared model/service files from `/Marks` (`Bookmark`, `ServerConfig`, `LinkdingAPI`, `MarksAuth`, `Log`, `PodcastRequests`).
|
||||
- `ShareExtension` — app extension with `/ShareExtension` plus selected shared model/service files from `/Marks` (`Bookmark`, `IngestPayload`, `ServerConfig`, `LinkdingAPI`, `MarksAuth`, `Log`, `PodcastRequests`).
|
||||
- `MarksWidget` — WidgetKit extension with `/MarksWidget`.
|
||||
- `MarksTests` — unit-test bundle depending on `Marks`.
|
||||
|
||||
@@ -22,7 +22,8 @@ At startup:
|
||||
1. `MarksApp` loads a saved `ServerConfig` or falls back to a built-in default config.
|
||||
2. `MainContainer` constructs `LinkdingAPI(config:)`.
|
||||
3. `MainContainer` stores `BookmarksViewModel(api:cacheKey:)` in `@State`, keeping the model stable across SwiftUI re-renders.
|
||||
4. A top-level `TabView` presents Bookmarks, Tags, Podcasts, and Search.
|
||||
4. `MainContainer` also creates an `IngestedSourceLibrary` for local source storage.
|
||||
5. A top-level `TabView` presents Bookmarks, Tags, Sources, Podcasts, and Search.
|
||||
|
||||
`MainContainer` also handles:
|
||||
|
||||
@@ -70,6 +71,19 @@ Implemented operations include:
|
||||
|
||||
`Bookmark`, `BookmarkResponse`, create/update payloads, `BookmarkCheck`, tag response models, and `SmartCollection` live in `/Marks/Models/Bookmark.swift`.
|
||||
|
||||
## Local source ingest
|
||||
|
||||
Local source ingest is separate from linkding bookmarks. It is owned by:
|
||||
|
||||
- `/Marks/Models/IngestedSource.swift` — source metadata for text, PDF, and generic file imports.
|
||||
- `/Marks/Services/IngestedSourceStore.swift` — JSON metadata persistence, security-scoped file copying, PDFKit text extraction, plain-text extraction, and local search.
|
||||
- `/Marks/Services/SourceSpotlightIndexer.swift` — Spotlight indexing/removal for source title, body text, tags, kind, and original filename.
|
||||
- `/Marks/Views/SourcesView.swift` — Sources tab list/search, text import, file importer, QuickLook preview, delete, and podcast actions.
|
||||
|
||||
`IngestedSourceStore` writes `Sources/sources.json` and imported originals under `Sources/files/` in Application Support. Text imports store only metadata/body text. File imports copy the selected file into app storage; PDF imports extract text with `PDFDocument`, while other files attempt UTF-8/ASCII extraction.
|
||||
|
||||
Source podcasts use `IngestedSource.podcastURL` (`marks-source://{uuid}`) as the podcast cache/index key and send up to 100,000 characters of `bodyText` to the backend as `text`.
|
||||
|
||||
## Configuration and app-group sharing
|
||||
|
||||
`ServerConfig` contains linkding host, optional port, path, token, and HTTPS flag. Its extension stores config in two places:
|
||||
@@ -94,7 +108,7 @@ Security note: `/Marks/MarksApp.swift` currently includes a built-in default con
|
||||
|
||||
`/Marks/Services/AnalyticsService.swift` sends events through the backend path used by `ClaudeService`/`MarksAuth`.
|
||||
|
||||
Podcast audio is downloaded to Application Support under a `podcasts` directory using a stable hash of the article URL. `PodcastIndex` records metadata in `index.json`; `PodcastLibrary` reads that index for the Podcasts tab and widget metadata.
|
||||
Podcast generation accepts either a web URL or a local source pseudo-URL. `ClaudeService.generatePodcast(url:title:sourceText:sourceKind:)` always sends `url` and may also send `title`, `text`, and `source_kind`. Podcast audio is downloaded to Application Support under a `podcasts` directory using a stable hash of the article/source URL. `PodcastIndex` records metadata in `index.json`; `PodcastLibrary` reads that index for the Podcasts tab and widget metadata.
|
||||
|
||||
## On-device Ask and Spotlight
|
||||
|
||||
@@ -109,13 +123,13 @@ The app has two AI paths:
|
||||
- `SpotlightBookmarkSearch`/`BookmarkSearchTool` search the index for relevant bookmarks.
|
||||
- `BookmarkEntity` also conforms to indexed App Intents concepts for system-level entity discovery.
|
||||
|
||||
If changing bookmark fields, keep `Bookmark`, `BookmarkEntity`, Spotlight indexing, widget payloads, and row UI in sync.
|
||||
If changing bookmark fields, keep `Bookmark`, `BookmarkEntity`, Spotlight indexing, widget payloads, and row UI in sync. If changing source fields, keep `IngestedSource`, `IngestedSourceStore`, `SourceSpotlightIndexer`, Sources UI, and source podcast generation in sync.
|
||||
|
||||
## Share extension architecture
|
||||
|
||||
The share extension is implemented by `/ShareExtension/ShareViewController.swift` and `/ShareExtension/ShareView.swift`.
|
||||
|
||||
It accepts a shared URL/text payload, creates a SwiftUI save card, loads shared `ServerConfig`, checks linkding for an existing bookmark, and lets the user edit tags, notes, read-later state, and a Create Podcast toggle.
|
||||
It accepts shared URL, HTML, RTF, plain-text, and text payloads, uses `IngestPayloadParser` to find the first bookmarkable `http`/`https` URL and preserve non-URL notes, creates a SwiftUI save card, loads shared `ServerConfig`, checks linkding for an existing bookmark, and lets the user edit tags, notes, read-later state, and a Create Podcast toggle.
|
||||
|
||||
For suggestions, it combines:
|
||||
|
||||
@@ -123,7 +137,7 @@ For suggestions, it combines:
|
||||
- the user's existing tag vocabulary from `fetchTags()`,
|
||||
- backend AI tag suggestions through `/ShareExtension/TagSuggester.swift`.
|
||||
|
||||
Existing bookmarks are updated rather than duplicated. New bookmarks are created through `LinkdingAPI`. Podcast generation is not performed inside the extension; it queues a `PodcastRequests` item in the app group for the main app to process later.
|
||||
Existing bookmarks are updated rather than duplicated. New bookmarks are created through `LinkdingAPI`. Podcast generation is not performed inside the extension; it queues a `PodcastRequests` item in the app group for the main app to process later. The same parser is used by in-app Add Bookmark text import.
|
||||
|
||||
## Widget architecture and deep links
|
||||
|
||||
@@ -157,4 +171,4 @@ Widget links use custom URLs built by `URL.marksBookmark(_:)` and `URL.marksPodc
|
||||
|
||||
## Recent history context
|
||||
|
||||
Recent git history shows major investment in podcast UX and on-device AI: the latest merge added played/unplayed queues, background podcast generation, share-sheet audio requests, podcast library/index changes, and player improvements. A prior AI commit added on-device RAG, Spotlight indexing, and unified logging. Use those areas cautiously because they span app state, storage, widgets, share extension, and App Intents rather than a single view.
|
||||
Recent git history shows major investment in ingest and podcast UX: the latest change added local text/PDF/file sources, richer URL extraction for share/add flows, source Spotlight indexing, and source-backed podcast generation. Earlier podcast and AI changes added played/unplayed queues, background podcast generation, share-sheet audio requests, podcast library/index changes, player improvements, on-device RAG, Spotlight indexing, and unified logging. Use those areas cautiously because they span app state, storage, widgets, share extension, and App Intents rather than a single view.
|
||||
|
||||
Reference in New Issue
Block a user