> ## Documentation Index
> Fetch the complete documentation index at: https://docs.echos.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Knowledge Import

> Import Obsidian vaults, Notion exports, plain markdown files, PDFs, audio, and RSS feeds.

# Knowledge Import Guide

EchOS stores every note as a plain `.md` file with YAML frontmatter, making the knowledge base
natively readable in any markdown editor and trivially importable from Obsidian vaults,
Notion exports, PDFs, audio files, and RSS feeds.

***

## Capturing content from URLs

The fastest way to import content is to ask the agent directly. EchOS includes plugins for several content types:

| Content type    | Plugin                  | Example prompt                               |
| --------------- | ----------------------- | -------------------------------------------- |
| Web article     | `@echos/plugin-article` | *"Save this article: \[URL]"*                |
| YouTube video   | `@echos/plugin-youtube` | *"Save this YouTube video: \[URL]"*          |
| Tweet / thread  | `@echos/plugin-twitter` | *"Save this tweet: \[URL]"*                  |
| PDF document    | `@echos/plugin-pdf`     | *"Save this PDF: \[URL]"*                    |
| Podcast / audio | `@echos/plugin-audio`   | *"Transcribe and save this podcast: \[URL]"* |
| RSS / Atom feed | `@echos/plugin-rss`     | *"Subscribe to this RSS feed: \[URL]"*       |

Each plugin downloads the content, extracts the relevant text (or transcript), and saves it as a searchable markdown note. AI categorization is applied automatically when `ANTHROPIC_API_KEY` is configured.

### PDF import

```
Save this paper as a note: https://example.com/paper.pdf
```

The agent calls `save_pdf`, which:

1. Downloads the PDF
2. Extracts text via `pdf-parse`
3. Creates a note with `type: article` and `inputSource: file`

Password-protected or image-only PDFs fail gracefully with a descriptive message.

### Audio and podcast import

```
Transcribe and save this podcast episode: https://example.com/episode.mp3
```

The agent calls `save_audio`, which:

1. Probes the file size (HEAD, then `Range: bytes=0-0` if needed)
2. Downloads — splitting into 24 MB chunks if the file exceeds 25 MB
3. Transcribes each chunk via OpenAI Whisper (`whisper-1`)
4. Saves a note with `type: note` and `inputSource: voice`

**Supported formats:** `.mp3`, `.wav`, `.m4a`, `.ogg`, `.webm`, `.mp4`, `.flac`

**Requirement:** `OPENAI_API_KEY` must be configured. See [Setup](/setup-fixes) for details.

### RSS and Atom feed subscriptions

```
Subscribe to this RSS feed: https://example.com/feed.xml
```

The agent calls `manage_feeds` (action: `add`), which:

1. Validates the URL (SSRF protection)
2. Fetches the feed to confirm it is parseable
3. Stores the subscription in the plugin's SQLite database

New articles are then fetched automatically every 4 hours. For each entry the plugin:

1. Claims the article's guid atomically (deduplication)
2. Extracts the full article text via `@echos/plugin-article`
3. Applies AI categorization (inherits feed tags plus `rss`)
4. Saves a note with `type: article` and `inputSource: url`

**Manage subscriptions:**

```
List my RSS feeds
Refresh all my RSS feeds now
Unsubscribe from https://example.com/feed.xml
```

See [RSS Feed Plugin](/plugins#rss-feed-plugin) for full details.

***

## Opening EchOS in Obsidian

Point Obsidian at the knowledge directory and everything works immediately:

1. In Obsidian: **Open folder as vault** → select your `KNOWLEDGE_DIR` (defaults to `~/echos/knowledge/`, or wherever configured in your `.env`)
2. All notes open with their frontmatter properties visible in Obsidian's Properties panel
3. Live edits in Obsidian are picked up automatically by EchOS's file watcher — no restart needed

<Note>EchOS watches `KNOWLEDGE_DIR` for changes. Saving a file in Obsidian triggers a re-index within seconds. New files added via Obsidian are indexed the same way.</Note>

***

## Dropping in pre-formatted files

Any `.md` file that already has the required EchOS frontmatter is indexed automatically:

1. Copy/paste the file into any folder under your knowledge directory (defaults to `~/echos/knowledge/`, folder structure is free-form)
2. Either wait for the file watcher to pick it up (if EchOS is running), or run:
   ```bash theme={null}
   pnpm reconcile
   ```
3. The file appears in search results immediately

Required frontmatter fields are listed in the [Frontmatter reference](#frontmatter-reference) below.

***

## Importing from an Obsidian vault

Use the `import:obsidian` script to convert an existing Obsidian vault into EchOS format.

### What gets converted

| Obsidian field                          | EchOS field                   | Fallback                                  |
| --------------------------------------- | ----------------------------- | ----------------------------------------- |
| `id` (if UUID)                          | skipped — file already native | —                                         |
| `title` / `aliases[0]`                  | `title`                       | filename (date prefix stripped)           |
| `created` / `date` / `dateCreated`      | `created`                     | file birthtime                            |
| `updated` / `modified` / `dateModified` | `updated`                     | file mtime                                |
| `tags` (array or string)                | `tags`                        | inline `#tags` from content               |
| `category`                              | `category`                    | first path segment relative to vault root |
| `[[WikiLinks]]`                         | `links`                       | extracted from content body               |
| `status` (if valid)                     | `status`                      | omitted                                   |

### Step by step

**1. Preview first (recommended)**

```bash theme={null}
pnpm import:obsidian --source ~/my-obsidian-vault --dry-run
```

This prints what would happen without writing anything:

```
Found 847 markdown file(s) in /Users/you/my-obsidian-vault

  [convert] daily/2024-01-15 Morning pages.md
  [convert] projects/echos/Architecture ideas.md
  [skip]    templates/Daily Note Template.md (already has EchOS id)
  ...

Summary:
  Processed : 847
  Converted : 843
  Skipped   : 4
  Errors    : 0
```

**2. Convert in place** (modifies the vault files directly)

```bash theme={null}
pnpm import:obsidian --source ~/my-obsidian-vault
```

Each file gets EchOS frontmatter written back into it. Original content is preserved.

**3. Convert to a new location** (non-destructive)

```bash theme={null}
pnpm import:obsidian \
  --source ~/my-obsidian-vault \
  --target $ECHOS_HOME/knowledge \
  --copy
```

Files are copied to `$ECHOS_HOME/knowledge/{type}/{category}/{date}-{slug}.md` (defaults to `~/echos/knowledge/`) with EchOS frontmatter.

**4. Override the default type**

```bash theme={null}
pnpm import:obsidian --source ~/vault --type journal
```

All notes without an explicit `type` in their frontmatter will be imported as that type.

Valid types: `note` | `journal` | `article` | `youtube` | `tweet` | `reminder` | `conversation` | `image`

**5. Force tags on every imported note**

```bash theme={null}
pnpm import:obsidian --source ~/vault --tags ai-corner,article
```

The tags are **merged** (not replaced) with any tags already in the file's frontmatter or inline
`#hashtags`. Useful when importing a folder of thematically related files that have no frontmatter.

**6. Override category for every imported note**

```bash theme={null}
pnpm import:obsidian --source ~/vault --category articles
```

Overrides the category that would normally be inferred from the vault's directory structure.

**7. Re-process already-imported files (`--force`)**

If you ran the importer before and files already have an EchOS `id`, they are skipped by default.
Use `--force` to re-process them: tags are merged in, `--category` / `--type` overrides are applied,
and the existing `id` (and all other fields) is preserved.

```bash theme={null}
# Add tags + fix category on files that were already imported
pnpm import:obsidian \
  --source ~/Documents/ai-corner \
  --tags ai-corner,article \
  --category articles \
  --force

pnpm reconcile
```

`--force` works with `--copy` too: the patched file is written to the target directory.

**Combining flags — real-world example**

Importing a fresh folder of AI articles (no frontmatter), tagging them consistently and
copying to the knowledge base without touching the originals:

```bash theme={null}
pnpm import:obsidian \
  --source ~/Documents/ai-corner \
  --type article \
  --tags ai-corner,article \
  --category articles \
  --copy

pnpm reconcile
```

**7. Index the imported notes**

```bash theme={null}
pnpm reconcile
```

Or just restart EchOS — it reconciles on startup.

***

## Importing from Notion

Notion's markdown export has a few quirks the `import:notion` script handles automatically:

* **Filename UUID suffix** — Notion appends a short hex ID: `My Note abc123def456.md` → title becomes `My Note`
* **Date formats** — `Created: January 1, 2023` or ISO 8601; script tries multiple field names
* **Comma-separated tags** — `Tags: ai, productivity` → `['ai', 'productivity']`
* **No type field** — all imported as `'note'`

### Notion export format

To export from Notion: **Settings → Export content → Markdown & CSV → Export**.
Unzip the export. The folder contains one `.md` per page plus sub-folders for nested pages.

### Step by step

**1. Preview**

```bash theme={null}
pnpm import:notion --source ~/Downloads/notion-export --dry-run
```

**2. Convert and write to EchOS**

```bash theme={null}
pnpm import:notion \
  --source ~/Downloads/notion-export \
  --target ~/echos/knowledge
```

Files are always written to `--target` (not modified in place), at:
`{target}/note/{category}/{date}-{slug}.md`

**3. Index**

```bash theme={null}
pnpm reconcile
```

### Before/after example

**Notion export (`My Ideas abc123.md`):**

```yaml theme={null}
---
Created: January 1, 2023
Last Edited Time: March 15, 2023
Tags: ai, tools, productivity
Category: Research
---

# My Ideas

Some content here...
```

**After `import:notion`:**

```yaml theme={null}
---
id: f47ac10b-58cc-4372-a567-0e02b2c3d479
type: note
title: My Ideas
created: '2023-01-01T00:00:00.000Z'
updated: '2023-03-15T00:00:00.000Z'
tags:
  - ai
  - tools
  - productivity
links: []
category: Research
---

# My Ideas

Some content here...
```

***

## Frontmatter reference

All fields EchOS reads from `.md` files:

| Field         | Required | Type       | Valid values / notes                                                                                                             |
| ------------- | -------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `id`          | **Yes**  | `string`   | UUID v4 — generated by import scripts                                                                                            |
| `type`        | **Yes**  | `string`   | `note` \| `journal` \| `article` \| `youtube` \| `tweet` \| `reminder` \| `conversation` \| `image`                              |
| `title`       | **Yes**  | `string`   | Free text                                                                                                                        |
| `created`     | **Yes**  | `string`   | ISO 8601 datetime, e.g. `'2024-01-15T09:30:00.000Z'`                                                                             |
| `updated`     | **Yes**  | `string`   | ISO 8601 datetime                                                                                                                |
| `tags`        | **Yes**  | `string[]` | YAML list; may be empty `[]`                                                                                                     |
| `links`       | **Yes**  | `string[]` | Related note titles or IDs; may be empty `[]`                                                                                    |
| `category`    | **Yes**  | `string`   | Free text, e.g. `programming`, `health`                                                                                          |
| `status`      | No       | `string`   | `saved` \| `read` \| `archived`                                                                                                  |
| `inputSource` | No       | `string`   | `text` \| `voice` \| `url` \| `file` \| `image` — audio/podcast notes use `voice`; PDF notes use `file`; image notes use `image` |
| `source_url`  | No       | `string`   | Original URL for articles/YouTube notes                                                                                          |
| `gist`        | No       | `string`   | Short summary / AI-generated abstract                                                                                            |
| `author`      | No       | `string`   | Author name for articles                                                                                                         |

<Warning>Files missing `id` are silently skipped by the reconciler. The import scripts ensure `id` is always written. If you create notes manually, you must add a UUID `id` field.</Warning>

***

## After import: run `pnpm reconcile`

```bash theme={null}
pnpm reconcile
```

This syncs all markdown files in `KNOWLEDGE_DIR` into:

* **SQLite** (metadata + FTS5 full-text index)
* **LanceDB** (placeholder vector embeddings — real embeddings generated when EchOS runs)

Output looks like:

```
Results:
  Scanned : 850
  Added   : 843
  Updated : 0
  Skipped : 7
  Deleted : 0
```

Alternatively, just start EchOS (`pnpm start`) — it reconciles on every startup.