How I Wake Up to Fresh YouTube Transcripts in Obsidian

A research intake system, not a YouTube summarizer

This is not a YouTube summarizer. It is a research intake system. Summarizers help when you already found the video. This workflow finds the new material overnight, files it, summarizes it, and leaves you with a morning research inbox.

A single YouTube summary is useful once. A research inbox is useful every morning. The difference is not the model. It is the system around it: trusted channels, scheduled checks, markdown transcripts, summaries, and a folder Claude can search later.

Use this instead of

If your need is in the top two rows, this workflow is overkill. If it’s in the bottom two, the manual version collapses the moment you care about more than five channels.

If you need	Use
One-off summary of a video you’re already watching	Chrome extension or Claude skill
Structured notes from one pasted URL	Obsidian YouTube plugin or Claude skill
A repeated weekly research stream from trusted sources	This cron + transcript + summary pipeline
A searchable source archive for future Claude or Cursor work	This markdown folder workflow

Who this is for

Designers, researchers, content people, and design system leads who use YouTube as a learning source but do not want to manually watch, bookmark, summarize, and file every useful video.

You do not need a perfect media pipeline. The point is simpler:

• pick the channels that matter
• check them automatically
• pull the latest transcripts
• save them as plain markdown
• let AI summarize them
• sync the results into a knowledge base you actually use

The problem

YouTube is full of useful design, AI, engineering, product, and founder content. But the manual workflow is terrible.

You have to notice the video, decide if it is worth watching, open it, scrub around, maybe copy the transcript, maybe ask an AI tool to summarize it, then maybe save the notes somewhere. By the time you do all of that, the friction has already killed the habit.

I wanted the opposite workflow:

I go to sleep. The system checks the channels I care about. If there are new videos, it saves their transcripts, summarizes them, and puts everything in Obsidian. In the morning, I can scan what arrived and decide what deserves attention.

That turns YouTube from an infinite feed into a structured research stream.

The simple architecture

The flow has four parts:

1. A channel list
2. A transcript fetcher
3. A server-side summarizer
4. An Obsidian sync

The important design decision is that each part writes plain markdown files. That keeps the whole system easy to inspect, debug, move, and reuse with AI tools.

Here is the shape of the system:

YouTube channels
  -> nightly script checks latest videos
  -> transcript API saves markdown files
  -> remote server stores transcripts
  -> summarizer creates markdown summaries
  -> local sync pulls everything into Obsidian

This is deliberately boring. Boring is good here. A folder of markdown files is easier to trust than a clever dashboard you never maintain.

Step 1: Define the channels worth watching

The first file is just a list of channels.

Each channel has:

• a human-readable name
• the YouTube handle
• how many recent videos to check per run

Example:

CHANNELS=(
  "Lenny's Podcast|LennysPodcast|2"
)

This is the first useful constraint. I am not asking the system to understand all of YouTube. I am telling it which sources already have a high chance of being useful.

The last number is a safety valve. Most channels only need the latest one or two videos checked each day. That keeps the run fast and avoids pulling a huge backlog every morning.

Step 2: Use yt-dlp to find the latest videos

The script does not download the full video. It only asks YouTube for the latest video IDs and titles.

The command looks like this:

yt-dlp --flat-playlist --playlist-end "$LATEST" \
  --print "%(id)s|%(title)s" \
  "https://www.youtube.com/@${HANDLE}/videos"

The useful bit is --flat-playlist. It gets the metadata without downloading the media.

For each channel, the script returns rows like:

VIDEO_ID|Video title

That gives the rest of the workflow enough information to check whether the video is new and decide what file to create.

Step 3: Skip videos that already exist

Before fetching anything, the script checks the transcript folder on the server.

The rule is simple: if a markdown file already exists for that video ID, skip it.

That matters because scheduled jobs should be boring and repeatable. A daily job should be safe to run twice. It should not create duplicate transcripts just because a previous run partially completed.

The file naming pattern makes that possible:

YYYY-MM-DD_VIDEOID.md

Later, the local sync can rename files into something more readable, but the first save uses the video ID because IDs are stable.

Step 4: Fetch the transcript

For each new video, the script calls youtube-transcript-api from Python.

The basic idea:

from youtube_transcript_api import YouTubeTranscriptApi

api = YouTubeTranscriptApi()
transcript = api.fetch("VIDEO_ID")
snippets = [entry.text for entry in transcript]
full = "\n".join(snippets)

Then it writes a markdown file with a small metadata header:

<!-- model: yt-transcript-api -->
# Video title

**Channel:** Channel Name
**Video ID:** VIDEO_ID
**URL:** https://www.youtube.com/watch?v=VIDEO_ID
**Downloaded:** 2026-05-12

---

Transcript text...

This format is intentionally plain. I want the file to be readable in Obsidian, searchable from the terminal, and easy for Claude Code or Cursor to use later.

Step 5: Save transcripts on a server

The transcript files are copied to a small remote server.

The remote folder is organized by channel:

/YouTube/transcripts/
  AI-Engineer/
    2026-05-12_VIDEOID.md
  Greg-Isenberg/
    2026-05-12_VIDEOID.md
  Peter-Yang/
    2026-05-12_VIDEOID.md

I use the server as the overnight worker because it can run scheduled jobs even when I am not sitting at my laptop. It also keeps the pipeline separate from whatever project I happen to be working on locally.

For a public version of this guide, replace my actual server details with placeholders:

SERVER="user@your-server"
REMOTE_DIR="/path/to/YouTube/transcripts"

Do not publish real IP addresses, usernames, tokens, API keys, or private folder names.

Step 6: Trigger summaries when new transcripts arrive

If the run found new transcripts, it triggers a summarizer job on the server.

Conceptually:

if [ "$TOTAL_NEW" -gt 0 ]; then
  ssh "$SERVER" "nohup /path/to/youtube-summarize.sh > /path/to/logs/youtube-summarize-${DATE}.log 2>&1 &"
fi

The important part is nohup and the trailing &. The transcript fetcher does not need to sit there waiting for every summary to finish. It can hand off the work and exit.

The summarizer produces a second set of markdown files:

/YouTube/summaries/
  AI-Engineer/
    2026-05-12 Video title.md
  Greg-Isenberg/
    2026-05-12 Video title.md

I treat transcripts and summaries differently:

• transcripts are the raw source
• summaries are the morning scanning layer

If a summary looks useful, I can always open the transcript and pull exact quotes or details.

Step 7: Schedule the overnight jobs

The server has cron jobs for the overnight workflow.

The shape is:

01:00 - fetch YouTube transcripts
05:00 - summarize transcripts
08:15 - deliver summary notification
09:00 - local Obsidian sync

The exact times do not matter. What matters is the sequencing:

1. Fetch transcripts first.
2. Give summaries time to run.
3. Sync the finished files into Obsidian.
4. Start the day with a prepared research inbox.

If you are building this from scratch, start with one daily cron job. Add the rest only after the first piece is reliable.

Step 8: Sync into Obsidian

The morning sync pulls transcripts and summaries from the server into my Obsidian vault.

The local destination looks like this:

Obsidian vault/
  2 - YouTube/
    Transcripts/
    Summaries/

The sync uses rsync with --ignore-existing, so it only pulls new files:

rsync -avz --ignore-existing \
  "$SERVER:/path/to/YouTube/transcripts/" \
  "$OBSIDIAN_YT/Transcripts/"

rsync -avz --ignore-existing \
  "$SERVER:/path/to/YouTube/summaries/" \
  "$OBSIDIAN_YT/Summaries/"

After syncing, the script renames files that still use raw video IDs. It reads the title from the markdown heading and turns:

2026-05-12_VIDEOID.md

into:

2026-05-12 Video title.md

This makes the folder usable inside Obsidian. Video IDs are good for machines. Titles are better for humans.

What I wake up to

In the morning, I can open Obsidian and see:

• new transcripts by channel
• new summaries by date
• useful AI, product, design, and engineering videos collected without manual searching
• a searchable archive of source material
• markdown files that can be reused in Claude Code, Cursor, Substack drafts, scripts, or research notes

This is the real value. The automation does not decide what I should think. It prepares the material so I can think faster.

Source handling and ethics

A few rules I keep about handling the material this pipeline collects.

Store the source URLs. Every transcript file has the original YouTube URL in the metadata header. If you want to quote a creator’s actual words, go back to the video and check the published source. The transcript is a research artifact, not a citation.

Keep transcripts as private research. Auto-generated captions are not licensed for republication. Even human-edited transcripts belong to the channel. Treat the folder as your notebook, not a publishing surface.

Quote carefully. When something from a transcript ends up in a Substack post or a client deliverable, quote a short excerpt, attribute the creator and the video, link the original. Same rule you’d apply to a book.

Don’t republish transcripts. Posting full transcripts on a blog or feeding them into a public dataset is the move that gets pipelines like this shut down for everyone. The point is private research.

If you can’t keep these rules, don’t build the pipeline. Use a public summarizer that handles attribution for you.

Where this breaks

A scheduled pipeline is only useful if it’s honest about its failure modes. This one has six. None of them are dealbreakers. All of them have shown up in my actual logs.

No captions available. Some channels turn off captions or only enable them after manual review. The script silently skips these unless you add a check. When youtube-transcript-api returns nothing, log the video ID and move on. Don’t crash the run because one channel went quiet.

Auto-generated captions are low quality. YouTube’s auto-captions get product names wrong, drop punctuation, and merge speakers. A transcript that reads “Cloud Sonnet won” is the auto-caption version of “Claude Sonnet 4.6.” Run the summarizer on top, and you’ll get garbage summaries. The mitigation: prefer channels with human-edited captions. Mark auto-caption transcripts in the frontmatter so you can filter them out of downstream prompts.

YouTube blocks or changes access. YouTube has rate-limited or temporarily blocked yt-dlp twice in the last year. When it happens, your overnight run gets nothing. The fix is not heroic. Add a retry with backoff, log the failure, assume some mornings will be empty. Don’t build the workflow around 100 percent uptime.

Duplicate video IDs after partial runs. If the transcript download succeeds but the file write fails, the next run thinks the video doesn’t exist and tries again. You end up with two transcripts for the same video. Use the video ID as the filename and check existence atomically. If it’s there, skip.

The summary job fails after the transcript succeeds. The transcript writes fine. Then the summarizer fails because the model API timed out, or the prompt template broke, or your API key rotated. The transcript is on the server, but no summary appears in the morning. Add a separate retry job for failed summaries, decoupled from the transcript fetch. They are different failure modes and should be resilient independently.

The system collects too much low-signal content. This is the silent killer. You add a channel because one video was great, then 19 of the next 20 videos are noise. The morning inbox bloats. The summaries become a chore. The fix is editorial, not technical. Review the channel list every quarter and remove anything where the signal-to-noise ratio has dropped. The pipeline is only as good as the source list.

What this costs to run

The transcript layer is essentially free. The real cost is the summarizer.

Rough numbers for ten videos in one run (about 5,000 to 8,000 words of transcript each):

• yt-dlp: free
• youtube-transcript-api: free
• Summaries with Claude Sonnet 4.6: about $0.25 to $0.35 for ten videos
• Summaries with Claude Haiku 4.5: about $0.08 to $0.12 for ten videos
• Summaries with Gemini 2.5 Flash: about $0.03 to $0.05 for ten videos
• Small VPS for the overnight worker (Hetzner, Vultr): about $5 to $7 per month
• Obsidian, cron, rsync: free

If you process ten new videos every weekday, that adds up to roughly $1 to $7 per month for AI summaries, plus the VPS. Most months, the whole pipeline costs less than a single coffee.

The interesting part is the unit economics. The marginal cost of one more channel is close to zero. Once the pipeline runs, adding the next ten channels does not change your monthly bill in any meaningful way. That is what makes the system feel free even though it is not.

Why this works well with AI tools

AI tools are much better when the source material is already structured.

A random YouTube tab gives an AI assistant very little to work with. A folder of markdown transcripts gives it:

• titles
• dates
• channels
• source URLs
• full transcript text
• summaries
• stable filenames

That means I can ask better questions later:

• “Find the best examples about agent workflows from this week.”
• “Compare what Boris Cherny and Matt Pocock said about AI coding.”
• “Turn the last five relevant summaries into Substack ideas.”
• “Pull design system lessons from the AI Engineer videos.”

The overnight system is not just a convenience. It is context preparation.