crashing from wav file size i think

This commit is contained in:
2026-04-08 00:48:05 -06:00
parent 56be227245
commit 38ca9cfbad
5 changed files with 303 additions and 120 deletions

View File

@ -6,59 +6,59 @@ Features are grouped by priority. Check off items as they are implemented.
## 🔴 High Priority — Core editing gaps ## 🔴 High Priority — Core editing gaps
- [x] **Cut / Mute sections** — select a time range and choose to cut (remove entirely) or mute (silence audio while video continues). Cut sections show as red overlays, mute sections as transparent blue overlays on the timeline over the transcript text and audio waveform. Backend: `ffmpeg -af volume=0` for mute, time-based cutting for removal. 1. [x] **Cut / Mute sections** — select a time range and choose to cut (remove entirely) or mute (silence audio while video continues). Cut sections show as red overlays, mute sections as transparent blue overlays on the timeline over the transcript text and audio waveform. Backend: `ffmpeg -af volume=0` for mute, time-based cutting for removal.
- [x] **Silence / pause trimmer** — detect pauses using min duration (ms) + amplitude threshold (dB), then apply detected pauses as cut ranges. Initial endpoint: `/audio/detect-silence`; UI includes filter controls and an "Apply As Cuts" action. 2. [x] **Silence / pause trimmer** — detect pauses using min duration (ms) + amplitude threshold (dB), then apply detected pauses as cut ranges. Initial endpoint: `/audio/detect-silence`; UI includes filter controls and an "Apply As Cuts" action.
- [ ] **Operation-level undo for batch actions** — explicit undo entry for actions like "Apply Silence Trim" so one shortcut/click reverts the whole operation, while still allowing normal fine-grained undo/redo steps. 3. [x] **Operation-level undo for batch actions** — explicit undo entry for actions like "Apply Silence Trim" so one shortcut/click reverts the whole operation, while still allowing normal fine-grained undo/redo steps.
- [ ] **Grouped silence-trim zones (editable batch)** — when pauses are applied, tag them as a batch (`trim_group_id`) so the user can: (1) delete all zones from that auto-trim pass at once, and (2) still select/resize/delete individual zones independently. 4. [ ] **Grouped silence-trim zones (editable batch)** — when pauses are applied, tag them as a batch (`trim_group_id`) so the user can: (1) delete all zones from that auto-trim pass at once, and (2) still select/resize/delete individual zones independently.
- [ ] **Edit silence-trim group settings after apply** — allow reopening a trim group and changing its detection settings (min pause ms, threshold dB, pre/post buffers), then reapplying updates to that group without affecting unrelated edits. 5. [ ] **Edit silence-trim group settings after apply** — allow reopening a trim group and changing its detection settings (min pause ms, threshold dB, pre/post buffers), then reapplying updates to that group without affecting unrelated edits.
- [ ] **Volume / gain control** — per-selection or global audio gain slider. Every editor has this. Descript users constantly complain it's missing. Backend: `ffmpeg -af volume=Xdb`. 6. [ ] **Volume / gain control** — per-selection or global audio gain slider. Every editor has this. Descript users constantly complain it's missing. Backend: `ffmpeg -af volume=Xdb`.
- [ ] **Speed adjustment** — slow down or speed up a selection or the whole clip. Backend: `ffmpeg -filter:v setpts` + `atempo`. Common use case: slightly speed up boring sections. 7. [ ] **Speed adjustment** — slow down or speed up a selection or the whole clip. Backend: `ffmpeg -filter:v setpts` + `atempo`. Common use case: slightly speed up boring sections.
- [ ] **Cut preview** — before committing a delete, play what the audio will sound like with that section removed (pre-listen across the edit point). Pure frontend using Web Audio API — splice the AudioBuffer and play the join. 8. [ ] **Cut preview** — before committing a delete, play what the audio will sound like with that section removed (pre-listen across the edit point). Pure frontend using Web Audio API — splice the AudioBuffer and play the join.
- [ ] **Timeline shows output length** — deleted regions should visually collapse (or show as narrow gaps) so the user sees the *output* duration, not just the source duration. 9. [ ] **Timeline shows output length** — deleted regions should visually collapse (or show as narrow gaps) so the user sees the *output* duration, not just the source duration.
--- ---
## 🟡 Medium Priority — Widely expected features ## 🟡 Medium Priority — Widely expected features
- [ ] **Transcript search (Ctrl+F)** — find words/phrases in the transcript and highlight matches. Pure frontend. Critical for long-form content. Jump between matches with Enter. 10. [ ] **Transcript search (Ctrl+F)** — find words/phrases in the transcript and highlight matches. Pure frontend. Critical for long-form content. Jump between matches with Enter.
- [ ] **Mark In / Out + delete (I / O keys)** — keyboard shortcuts to mark a time range on the timeline, then delete it. Faster than click-dragging words. Store the in/out points in state, `Delete` removes them. 11. [ ] **Mark In / Out + delete (I / O keys)** — keyboard shortcuts to mark a time range on the timeline, then delete it. Faster than click-dragging words. Store the in/out points in state, `Delete` removes them.
- [ ] **Low-confidence word highlighting** — WhisperX already returns `confidence` per word. Words below a threshold (e.g. < 0.6) should be visually underlined or tinted so the user knows where to double-check. 12. [ ] **Low-confidence word highlighting** — WhisperX already returns `confidence` per word. Words below a threshold (e.g. < 0.6) should be visually underlined or tinted so the user knows where to double-check.
- [ ] **Re-transcribe selection** if Whisper gets a section wrong, let the user select a word range and re-run transcription on just that segment (optionally with a different model or language). 13. [ ] **Re-transcribe selection** if Whisper gets a section wrong, let the user select a word range and re-run transcription on just that segment (optionally with a different model or language).
- [ ] **Word text correction** allow editing the transcript text of a word without affecting its timing. Whisper gets homophones/proper nouns wrong constantly. Pure frontend state change; no backend needed. 14. [ ] **Word text correction** allow editing the transcript text of a word without affecting its timing. Whisper gets homophones/proper nouns wrong constantly. Pure frontend state change; no backend needed.
- [ ] **Named timeline markers** drop named marker pins on the waveform (like Resolve markers). Store as `{ id, time, label, color }` in the project. Rendered as colored triangles on the timeline canvas. 15. [ ] **Named timeline markers** drop named marker pins on the waveform (like Resolve markers). Store as `{ id, time, label, color }` in the project. Rendered as colored triangles on the timeline canvas.
- [ ] **Chapters** group markers into named chapter ranges. Useful for podcasts and lectures. Exportable as YouTube chapter timestamps in the description. 16. [ ] **Chapters** group markers into named chapter ranges. Useful for podcasts and lectures. Exportable as YouTube chapter timestamps in the description.
--- ---
## 🟢 Lower Priority — Differentiating / power features ## 🟢 Lower Priority — Differentiating / power features
- [ ] **Audio normalization / loudness targeting** single "Normalize" button that targets a LUFS level (-14 for YouTube, -16 for Spotify). Backend: `ffmpeg -af loudnorm`. Very high value for podcasters, ~23 hours of work. 17. [ ] **Audio normalization / loudness targeting** single "Normalize" button that targets a LUFS level (-14 for YouTube, -16 for Spotify). Backend: `ffmpeg -af loudnorm`. Very high value for podcasters, ~23 hours of work.
- [ ] **Background music track** a second audio track for background music with volume ducking. Major gap in Descript that TalkEdit could own. Backend: `ffmpeg` amix + `asendcmd` for auto-ducking. 18. [ ] **Background music track** a second audio track for background music with volume ducking. Major gap in Descript that TalkEdit could own. Backend: `ffmpeg` amix + `asendcmd` for auto-ducking.
- [ ] **Video zoom / punch-in** scale and position the video (crop, zoom, pan). Used constantly on talking-head videos for emphasis. Backend: `ffmpeg -vf crop/scale/zoompan`. 19. [ ] **Video zoom / punch-in** scale and position the video (crop, zoom, pan). Used constantly on talking-head videos for emphasis. Backend: `ffmpeg -vf crop/scale/zoompan`.
- [ ] **Multi-clip / append** load a second video and append it to the timeline. Even without a full multi-track timeline, "append clip" is a heavily used workflow. 20. [ ] **Multi-clip / append** load a second video and append it to the timeline. Even without a full multi-track timeline, "append clip" is a heavily used workflow.
- [ ] **Clip thumbnail strip** video frame thumbnails along the timeline so users can navigate visually, not only by waveform. Backend: `ffmpeg` thumbnail extraction at regular intervals. 21. [ ] **Clip thumbnail strip** video frame thumbnails along the timeline so users can navigate visually, not only by waveform. Backend: `ffmpeg` thumbnail extraction at regular intervals.
- [ ] **Batch silence removal** full-file scan + remove all pauses above threshold in one click. Distinct from the manual trimmer above; this is a "fix the whole file" operation. 22. [ ] **Batch silence removal** full-file scan + remove all pauses above threshold in one click. Distinct from the manual trimmer above; this is a "fix the whole file" operation.
- [ ] **Export to transcript text / SRT only** some users just want a clean `.txt` or `.srt` of the edited transcript without rendering video. 23. [ ] **Export to transcript text / SRT only** some users just want a clean `.txt` or `.srt` of the edited transcript without rendering video.
--- ---
@ -66,28 +66,28 @@ Features are grouped by priority. Check off items as they are implemented.
These aren't features to build they're things to make more visible in the UI and README: These aren't features to build they're things to make more visible in the UI and README:
- **100% offline / no account required** CapCut requires login and sends data to servers. Descript is cloud-first. TalkEdit never leaves the machine. 24. **100% offline / no account required** CapCut requires login and sends data to servers. Descript is cloud-first. TalkEdit never leaves the machine.
- **Local AI models** Ollama support means no API costs and no data leaving the device. 25. **Local AI models** Ollama support means no API costs and no data leaving the device.
- **Word-level precision** editing by deleting words (not dragging razor cuts) is faster for talking-head content than any timeline-based editor. 26. **Word-level precision** editing by deleting words (not dragging razor cuts) is faster for talking-head content than any timeline-based editor.
- **Works on long files** virtualized transcript + chunked waveform handles 1hr+ content that bogs down CapCut. 27. **Works on long files** virtualized transcript + chunked waveform handles 1hr+ content that bogs down CapCut.
--- ---
## ✅ Already Implemented ## ✅ Already Implemented
- Word-level transcript editing (select, drag, shift-click, delete) 28. Word-level transcript editing (select, drag, shift-click, delete)
- Ctrl+click word seek timeline to that position 29. Ctrl+click word seek timeline to that position
- Waveform timeline with zoom (Ctrl+scroll), scroll, drag-to-scrub playhead 30. Waveform timeline with zoom (Ctrl+scroll), scroll, drag-to-scrub playhead
- Auto-scroll waveform when playhead goes off-screen 31. Auto-scroll waveform when playhead goes off-screen
- AI filler word detection and removal (Ollama / OpenAI / Claude) 32. AI filler word detection and removal (Ollama / OpenAI / Claude)
- AI clip suggestions for social media 33. AI clip suggestions for social media
- Noise reduction (DeepFilterNet or FFmpeg ANLMDN) 34. Noise reduction (DeepFilterNet or FFmpeg ANLMDN)
- Export: fast stream-copy or full reencode (MP4/MOV/WebM, 720p/1080p/4K) 35. Export: fast stream-copy or full reencode (MP4/MOV/WebM, 720p/1080p/4K)
- Captions: SRT, VTT, ASS burn-in with font/color/position options 36. Captions: SRT, VTT, ASS burn-in with font/color/position options
- Speaker diarization 37. Speaker diarization
- Project save / load (.aive JSON format) 38. Project save / load (.aive JSON format)
- Undo / redo (100-level history via Zundo) 39. Undo / redo (100-level history via Zundo)
- Multi-format input (MP4, MKV, MOV, AVI, WebM, M4A) 40. Multi-format input (MP4, MKV, MOV, AVI, WebM, M4A)
- Keyboard shortcuts (Space, J/K/L, arrows, Ctrl+Z/Shift+Z, Ctrl+S, Ctrl+E) 41. Keyboard shortcuts (Space, J/K/L, arrows, Ctrl+Z/Shift+Z, Ctrl+S, Ctrl+E)
- Settings panel: AI provider config (Ollama, OpenAI, Claude) 42. Settings panel: AI provider config (Ollama, OpenAI, Claude)
- Cut/mute range creation on timeline with draggable zone edits and Delete-to-remove 43. Cut/mute range creation on timeline with draggable zone edits and Delete-to-remove

View File

@ -1,6 +1,6 @@
import { useState } from 'react'; import { useState, useEffect } from 'react';
import { useEditorStore } from '../store/editorStore'; import { useEditorStore } from '../store/editorStore';
import { Loader2, Scissors } from 'lucide-react'; import { Loader2, Scissors, Trash2, Play, Pause } from 'lucide-react';
type SilenceRange = { type SilenceRange = {
start: number; start: number;
@ -8,14 +8,31 @@ type SilenceRange = {
duration: number; duration: number;
}; };
type TrimAction = 'cut' | 'mute';
export default function SilenceTrimmerPanel() { export default function SilenceTrimmerPanel() {
const { videoPath, backendUrl, addCutRange, duration } = useEditorStore(); const {
videoPath,
backendUrl,
addCutRange,
addMuteRange,
removeCutRange,
removeMuteRange,
cutRanges,
muteRanges,
duration,
pauseUndo,
resumeUndo
} = useEditorStore();
const [minSilenceMs, setMinSilenceMs] = useState(500); const [minSilenceMs, setMinSilenceMs] = useState(500);
const [silenceDb, setSilenceDb] = useState(-35); const [silenceDb, setSilenceDb] = useState(-35);
const [preBufferMs, setPreBufferMs] = useState(80); const [preBufferMs, setPreBufferMs] = useState(80);
const [postBufferMs, setPostBufferMs] = useState(120); const [postBufferMs, setPostBufferMs] = useState(120);
const [isDetecting, setIsDetecting] = useState(false); const [isDetecting, setIsDetecting] = useState(false);
const [ranges, setRanges] = useState<SilenceRange[]>([]); const [ranges, setRanges] = useState<SilenceRange[]>([]);
const [trimAction, setTrimAction] = useState<TrimAction>('cut');
const [isActive, setIsActive] = useState(false);
const detectSilence = async () => { const detectSilence = async () => {
if (!videoPath) return; if (!videoPath) return;
@ -58,6 +75,9 @@ export default function SilenceTrimmerPanel() {
}; };
const applyAsCuts = () => { const applyAsCuts = () => {
// Pause undo tracking to group all cuts into a single undo operation
pauseUndo();
const preBufferSeconds = preBufferMs / 1000; const preBufferSeconds = preBufferMs / 1000;
const postBufferSeconds = postBufferMs / 1000; const postBufferSeconds = postBufferMs / 1000;
const maxEnd = duration > 0 ? duration : Number.POSITIVE_INFINITY; const maxEnd = duration > 0 ? duration : Number.POSITIVE_INFINITY;
@ -67,11 +87,71 @@ export default function SilenceTrimmerPanel() {
const start = Math.max(0, r.start + preBufferSeconds); const start = Math.max(0, r.start + preBufferSeconds);
const end = Math.min(maxEnd, r.end - postBufferSeconds); const end = Math.min(maxEnd, r.end - postBufferSeconds);
if (end - start >= 0.01) { if (end - start >= 0.01) {
addCutRange(start, end); if (trimAction === 'cut') {
addCutRange(start, end);
} else {
addMuteRange(start, end);
}
} }
} }
// Resume undo tracking - this creates a single undo entry for the entire batch
resumeUndo();
}; };
const removeExistingTrims = () => {
pauseUndo();
try {
// Remove all cut ranges that match detected silence ranges
cutRanges.forEach(range => {
ranges.forEach(silenceRange => {
if (Math.abs(range.start - silenceRange.start) < 0.1 &&
Math.abs(range.end - silenceRange.end) < 0.1) {
removeCutRange(range.id);
}
});
});
// Remove all mute ranges that match detected silence ranges
muteRanges.forEach(range => {
ranges.forEach(silenceRange => {
if (Math.abs(range.start - silenceRange.start) < 0.1 &&
Math.abs(range.end - silenceRange.end) < 0.1) {
removeMuteRange(range.id);
}
});
});
} finally {
resumeUndo();
}
};
const toggleActive = () => {
setIsActive(!isActive);
if (!isActive) {
// When activating, detect silence and apply
detectSilence().then(() => {
if (ranges.length > 0) {
applyAsCuts();
}
});
} else {
// When deactivating, remove existing trims
removeExistingTrims();
}
};
// Auto-detect when video changes
useEffect(() => {
if (videoPath && isActive) {
detectSilence().then(() => {
if (ranges.length > 0) {
applyAsCuts();
}
});
}
}, [videoPath]);
return ( return (
<div className="p-4 space-y-4"> <div className="p-4 space-y-4">
<div className="space-y-1"> <div className="space-y-1">
@ -142,43 +222,95 @@ export default function SilenceTrimmerPanel() {
</div> </div>
</div> </div>
<button <div className="space-y-1.5">
onClick={detectSilence} <label className="text-[11px] text-editor-text-muted font-medium">
disabled={isDetecting || !videoPath} Trim Action
className="w-full flex items-center justify-center gap-2 px-4 py-2.5 bg-editor-accent hover:bg-editor-accent-hover disabled:opacity-50 rounded-lg text-sm font-medium transition-colors" </label>
> <select
{isDetecting ? ( value={trimAction}
<> onChange={(e) => setTrimAction(e.target.value as TrimAction)}
<Loader2 className="w-4 h-4 animate-spin" /> className="w-full px-2.5 py-1.5 text-xs bg-editor-surface border border-editor-border rounded focus:border-editor-accent focus:outline-none"
Detecting pauses... >
</> <option value="cut">Cut (remove silence)</option>
) : ( <option value="mute">Mute (silence audio)</option>
'Detect Pauses' </select>
)}
</button>
</div>
{ranges.length > 0 && (
<div className="space-y-2">
<div className="flex items-center justify-between">
<span className="text-xs font-medium">Detected {ranges.length} pause ranges</span>
<button
onClick={applyAsCuts}
className="flex items-center gap-1 px-2 py-1 text-xs bg-editor-accent/20 text-editor-accent rounded hover:bg-editor-accent/30"
>
<Scissors className="w-3 h-3" />
Apply As Cuts
</button>
</div>
<div className="max-h-56 overflow-y-auto space-y-1 pr-1">
{ranges.slice(0, 50).map((r, i) => (
<div key={`${r.start}-${r.end}-${i}`} className="px-2 py-1.5 rounded bg-editor-surface border border-editor-border text-xs">
{r.start.toFixed(2)}s - {r.end.toFixed(2)}s ({r.duration.toFixed(2)}s)
</div>
))}
</div>
</div> </div>
)}
<div className="flex items-center justify-between">
<label className="text-[11px] text-editor-text-muted font-medium">
Active Mode
</label>
<button
onClick={toggleActive}
className={`flex items-center gap-2 px-3 py-1.5 text-xs rounded transition-colors ${
isActive
? 'bg-green-600 hover:bg-green-700 text-white'
: 'bg-editor-surface border border-editor-border hover:bg-editor-surface-hover'
}`}
>
{isActive ? (
<>
<Pause className="w-3 h-3" />
Active
</>
) : (
<>
<Play className="w-3 h-3" />
Inactive
</>
)}
</button>
</div>
<div className="flex gap-2">
<button
onClick={detectSilence}
disabled={isDetecting || !videoPath}
className="flex-1 flex items-center justify-center gap-2 px-3 py-2 bg-editor-accent hover:bg-editor-accent-hover disabled:opacity-50 rounded text-xs font-medium transition-colors"
>
{isDetecting ? (
<>
<Loader2 className="w-3 h-3 animate-spin" />
Detecting...
</>
) : (
'Detect Pauses'
)}
</button>
{ranges.length > 0 && (
<button
onClick={removeExistingTrims}
className="flex items-center gap-1 px-3 py-2 text-xs bg-red-600 hover:bg-red-700 text-white rounded transition-colors"
>
<Trash2 className="w-3 h-3" />
Remove Trims
</button>
)}
</div>
{ranges.length > 0 && (
<div className="space-y-2">
<div className="flex items-center justify-between">
<span className="text-xs font-medium">Detected {ranges.length} pause ranges</span>
<button
onClick={applyAsCuts}
className="flex items-center gap-1 px-2 py-1 text-xs bg-editor-accent/20 text-editor-accent rounded hover:bg-editor-accent/30"
>
<Scissors className="w-3 h-3" />
Apply As {trimAction === 'cut' ? 'Cuts' : 'Mutes'}
</button>
</div>
<div className="max-h-56 overflow-y-auto space-y-1 pr-1">
{ranges.slice(0, 50).map((r, i) => (
<div key={`${r.start}-${r.end}-${i}`} className="px-2 py-1.5 rounded bg-editor-surface border border-editor-border text-xs">
{r.start.toFixed(2)}s - {r.end.toFixed(2)}s ({r.duration.toFixed(2)}s)
</div>
))}
</div>
</div>
)}
</div>
</div> </div>
); );
} }

View File

@ -5,24 +5,31 @@ import { Play, Pause, SkipBack, SkipForward, Volume2 } from 'lucide-react';
export default function VideoPlayer() { export default function VideoPlayer() {
const videoRef = useRef<HTMLVideoElement>(null); const videoRef = useRef<HTMLVideoElement>(null);
const audioRef = useRef<HTMLAudioElement>(null);
const videoUrl = useEditorStore((s) => s.videoUrl); const videoUrl = useEditorStore((s) => s.videoUrl);
const isPlaying = useEditorStore((s) => s.isPlaying); const isPlaying = useEditorStore((s) => s.isPlaying);
const duration = useEditorStore((s) => s.duration); const duration = useEditorStore((s) => s.duration);
const { seekTo, togglePlay } = useVideoSync(videoRef);
// Determine if this is an audio file based on the URL
const isAudioFile = videoUrl && (videoUrl.includes('.wav') || videoUrl.includes('.mp3') || videoUrl.includes('.m4a') || videoUrl.includes('.aac'));
const mediaRef = isAudioFile ? audioRef : videoRef;
const { seekTo, togglePlay } = useVideoSync(mediaRef as React.RefObject<HTMLVideoElement | HTMLAudioElement | null>);
const [displayTime, setDisplayTime] = useState(0); const [displayTime, setDisplayTime] = useState(0);
useEffect(() => { useEffect(() => {
const video = videoRef.current; const media = mediaRef.current;
if (!video) return; if (!media) return;
let raf = 0; let raf = 0;
const tick = () => { const tick = () => {
setDisplayTime(video.currentTime); setDisplayTime(media.currentTime);
raf = requestAnimationFrame(tick); raf = requestAnimationFrame(tick);
}; };
raf = requestAnimationFrame(tick); raf = requestAnimationFrame(tick);
return () => cancelAnimationFrame(raf); return () => cancelAnimationFrame(raf);
}, [videoUrl]); }, [videoUrl, mediaRef]);
const formatTime = (seconds: number) => { const formatTime = (seconds: number) => {
const m = Math.floor(seconds / 60); const m = Math.floor(seconds / 60);
@ -41,11 +48,11 @@ export default function VideoPlayer() {
const skip = useCallback( const skip = useCallback(
(delta: number) => { (delta: number) => {
const video = videoRef.current; const media = mediaRef.current;
if (!video) return; if (!media) return;
seekTo(Math.max(0, Math.min(duration, video.currentTime + delta))); seekTo(Math.max(0, Math.min(duration, media.currentTime + delta)));
}, },
[seekTo, duration], [seekTo, duration, mediaRef],
); );
if (!videoUrl) { if (!videoUrl) {
@ -59,13 +66,37 @@ export default function VideoPlayer() {
return ( return (
<div className="w-full h-full flex flex-col"> <div className="w-full h-full flex flex-col">
<div className="flex-1 flex items-center justify-center bg-black rounded-lg overflow-hidden min-h-0"> <div className="flex-1 flex items-center justify-center bg-black rounded-lg overflow-hidden min-h-0">
<video {isAudioFile ? (
ref={videoRef} <audio
src={videoUrl} ref={audioRef}
className="max-w-full max-h-full object-contain" src={videoUrl}
playsInline className="max-w-full max-h-full"
onClick={togglePlay} controls={false}
/> onClick={togglePlay}
onError={(e) => {
console.error('Audio load error:', e);
console.error('Audio src:', videoUrl);
}}
onLoadStart={() => console.log('Audio load start:', videoUrl)}
onLoadedData={() => console.log('Audio loaded data')}
onCanPlay={() => console.log('Audio can play')}
/>
) : (
<video
ref={videoRef}
src={videoUrl}
className="max-w-full max-h-full object-contain"
playsInline
onClick={togglePlay}
onError={(e) => {
console.error('Video load error:', e);
console.error('Video src:', videoUrl);
}}
onLoadStart={() => console.log('Video load start:', videoUrl)}
onLoadedData={() => console.log('Video loaded data')}
onCanPlay={() => console.log('Video can play')}
/>
)}
</div> </div>
<div className="pt-2 space-y-1.5 shrink-0"> <div className="pt-2 space-y-1.5 shrink-0">

View File

@ -1,7 +1,7 @@
import { useCallback, useRef, useEffect } from 'react'; import { useCallback, useRef, useEffect } from 'react';
import { useEditorStore } from '../store/editorStore'; import { useEditorStore } from '../store/editorStore';
export function useVideoSync(videoRef: React.RefObject<HTMLVideoElement | null>) { export function useVideoSync(videoRef: React.RefObject<HTMLVideoElement | HTMLAudioElement | null>) {
const rafRef = useRef<number>(0); const rafRef = useRef<number>(0);
const { const {
setCurrentTime, setCurrentTime,
@ -52,13 +52,13 @@ export function useVideoSync(videoRef: React.RefObject<HTMLVideoElement | null>)
}, [videoRef]); }, [videoRef]);
useEffect(() => { useEffect(() => {
const video = videoRef.current; const media = videoRef.current;
if (!video) return; if (!media) return;
const onTimeUpdate = () => { const onTimeUpdate = () => {
cancelAnimationFrame(rafRef.current); cancelAnimationFrame(rafRef.current);
rafRef.current = requestAnimationFrame(() => { rafRef.current = requestAnimationFrame(() => {
let t = video.currentTime; let t = media.currentTime;
// Skip over deleted ranges and cut ranges (handle overlapping/chained ranges) // Skip over deleted ranges and cut ranges (handle overlapping/chained ranges)
const allSkipRanges = [...deletedRanges, ...cutRanges]; const allSkipRanges = [...deletedRanges, ...cutRanges];
@ -79,19 +79,21 @@ export function useVideoSync(videoRef: React.RefObject<HTMLVideoElement | null>)
} }
if (skipCount > 0) { if (skipCount > 0) {
video.currentTime = t; media.currentTime = t;
return; return;
} }
// Mute/unmute based on mute ranges // Mute/unmute based on mute ranges (only for video elements)
let shouldMute = false; if ('muted' in media) {
for (const range of muteRanges) { let shouldMute = false;
if (t >= range.start && t < range.end) { for (const range of muteRanges) {
shouldMute = true; if (t >= range.start && t < range.end) {
break; shouldMute = true;
break;
}
} }
media.muted = shouldMute;
} }
video.muted = shouldMute;
setCurrentTime(t); setCurrentTime(t);
}); });
@ -99,18 +101,18 @@ export function useVideoSync(videoRef: React.RefObject<HTMLVideoElement | null>)
const onPlay = () => setIsPlaying(true); const onPlay = () => setIsPlaying(true);
const onPause = () => setIsPlaying(false); const onPause = () => setIsPlaying(false);
const onLoadedMetadata = () => setDuration(video.duration); const onLoadedMetadata = () => setDuration(media.duration);
video.addEventListener('timeupdate', onTimeUpdate); media.addEventListener('timeupdate', onTimeUpdate);
video.addEventListener('play', onPlay); media.addEventListener('play', onPlay);
video.addEventListener('pause', onPause); media.addEventListener('pause', onPause);
video.addEventListener('loadedmetadata', onLoadedMetadata); media.addEventListener('loadedmetadata', onLoadedMetadata);
return () => { return () => {
video.removeEventListener('timeupdate', onTimeUpdate); media.removeEventListener('timeupdate', onTimeUpdate);
video.removeEventListener('play', onPlay); media.removeEventListener('play', onPlay);
video.removeEventListener('pause', onPause); media.removeEventListener('pause', onPause);
video.removeEventListener('loadedmetadata', onLoadedMetadata); media.removeEventListener('loadedmetadata', onLoadedMetadata);
cancelAnimationFrame(rafRef.current); cancelAnimationFrame(rafRef.current);
}; };
}, [videoRef, deletedRanges, cutRanges, muteRanges, setCurrentTime, setIsPlaying, setDuration]); }, [videoRef, deletedRanges, cutRanges, muteRanges, setCurrentTime, setIsPlaying, setDuration]);

View File

@ -55,6 +55,8 @@ interface EditorActions {
getWordAtTime: (time: number) => number; getWordAtTime: (time: number) => number;
loadProject: (projectData: any) => void; loadProject: (projectData: any) => void;
reset: () => void; reset: () => void;
pauseUndo: () => void;
resumeUndo: () => void;
} }
const initialState: EditorState = { const initialState: EditorState = {
@ -327,6 +329,22 @@ export const useEditorStore = create<EditorState & EditorActions>()(
}, },
reset: () => set(initialState), reset: () => set(initialState),
pauseUndo: () => {
// Access the temporal store through the useEditorStore
const temporalStore = (useEditorStore as any).temporal;
if (temporalStore) {
temporalStore.getState().pause();
}
},
resumeUndo: () => {
// Access the temporal store through the useEditorStore
const temporalStore = (useEditorStore as any).temporal;
if (temporalStore) {
temporalStore.getState().resume();
}
},
}), }),
{ limit: 100 }, { limit: 100 },
), ),