Cutting 16:9 down to 9:16 discards roughly two thirds of the width, and whatever is left is the whole video as far as a phone viewer is concerned. The reframe engine detects faces across the footage, works out who is talking, and places the crop on them — holding still inside a shot and moving only when the camera actually does.
Real ClipSpeedAI output — every clip below was scored, cut, reframed to 9:16 and captioned automatically from the video above it.
Source video
15:27
Three stages, none of which need you to place a keyframe.
Paste a YouTube URL or upload the file. Source material can be a single locked-off camera, a multicam interview export, a screen recording with a webcam corner, or gameplay capture — the engine inspects what is there rather than assuming a format. Paid plans handle up to two hours per upload and the free demo takes anything under thirty minutes.
A face detector runs across the frames and the detections are linked into tracks so a person keeps the same identity from one frame to the next. Where several people are visible, audio is used to work out which one is speaking. The clip is then divided at camera changes, and each segment gets a single crop position derived from where the chosen subject actually sits in it.
Reframed clips come back ready to post, and you can still change your mind: switch layout, adjust the in and out points, or pick a different aspect ratio and re-render. Automation should get you to a good default fast, not lock you out of the decision.
Each of these exists because of a specific way that automatic cropping goes wrong.
Detection uses a YOLO-based face model rather than a simple frontal detector, because podcast and stream footage is full of the cases that break simple detectors: faces turned three-quarters away, faces at the edge of a wide shot, faces small in the frame, uneven lighting from a single key light. Missing a face is what causes a crop to sit on the wrong half of the screen for ten seconds.
Raw detections flicker — a box appears, vanishes for two frames, comes back slightly displaced. Detections are joined into continuous tracks so the system reasons about a person rather than about a box in a single frame. Without this layer, a crop jitters constantly and every viewer feels it even if they cannot say why.
Two people on screen means picking one. Choosing by size or centrality gives you the host every time, including through the guest's best answer. Audio is used to identify who is actually talking so the frame is on the person the viewer wants to see, which is the single largest difference between a tracked crop and a naive one.
People turn around, wear sunglasses, stand too far from the camera, or drop their head to read. When face detection comes up short across a segment, body and pose position is used instead, so the crop still lands on the human rather than reverting to the geometric middle of the frame. Reverting to centre is the failure that makes automated crops look unattended.
A crop that continuously chases small head movements produces a slow drifting wobble that is genuinely unpleasant to watch on a phone. The engine takes one position for each shot — the settled position of the subject through that segment — and holds it. When the camera cuts to a different angle, the crop snaps to the new subject rather than gliding across the frame.
Working out where the shots begin and end is harder than it sounds on multicam podcasts, where every camera shares the same room, lighting and colour, so a switch barely registers as a visual change. Face and subject data is used alongside scene detection precisely because the picture alone does not always announce a cut.
Sometimes the honest answer is that no single 9:16 window contains the shot. Split-screen stacks two people, three-up and four-up grids handle a panel, screenshare keeps a slide legible with the presenter inset, and gameplay picture-in-picture puts the webcam over the action. Seven layouts in total, selected from what is actually on screen.
Export 9:16 for Reels, TikTok and Shorts, 1:1 for feed placements, or 16:9 if you want a tighter landscape version of a wide original. The tracking work is shared; only the framing window changes, so you are not paying for the analysis three times.
The whole detection and tracking stack runs without a GPU. That is an unglamorous engineering choice with a practical consequence: capacity scales predictably and you are not queued behind somebody else's render, which matters most when clips are being cut from a live stream and lateness makes them worthless.
The classic wide two-shot, where the middle of the frame contains a table and two microphones. Also covered in depth on the podcast to Reels page.
A webcam box in a corner and the action everywhere else. Picture-in-picture keeps both instead of forcing a choice between your face and the play.
A presenter who paces, gestures at a board, and turns away mid-sentence. This is where the pose fallback earns its keep, because faces disappear constantly.
Filmed wide from the back of a room with the speaker small in frame. Tight tracking is the only way that footage becomes watchable on a phone.
You get whatever the client shot, in whatever condition. A reframe that copes with unpredictable source beats one tuned for a studio.
Years of landscape video that cannot be posted vertically without recomposition. The YouTube to TikTok converter covers that job end to end.
Cropping a 1920-pixel-wide frame to a 9:16 window leaves you roughly 600 pixels of width. Everything outside that is gone. If the window is fixed to the middle of the frame, then whatever the camera operator placed in the middle is what your viewers get — and on a two-person set, the middle is deliberately empty, because the composition put a person on each third.
The result is the most recognisable artefact of lazy vertical conversion: a video of a table, a plant and two disembodied shoulders, with an audible conversation happening off screen. It is not subtle, and it tells the viewer within half a second that nobody looked at this before posting it.
Even on a solo shot the centre is often wrong. Interview framing habitually places the subject off-centre with looking room in the direction they face. A centre crop takes the looking room and clips the ear. Tracking exists because the correct crop position is a property of the footage, not a constant.
The first pass is detection. A YOLO-based face model scans the footage and returns boxes with confidence values. This model choice is deliberate — lightweight frontal detectors do fine on a webcam looking straight down the lens and fall apart on the wide, angled, unevenly lit shots that real podcast and stage footage consists of.
The second pass is association. Boxes in consecutive frames get linked into tracks, which is what converts a stream of independent detections into people who persist through time. Alongside that, MediaPipe is used for lighter per-shot sampling — a handful of frames spread across a segment, checked for where the subject sits, rather than an expensive analysis of every single frame. When face detection returns too little across a segment, pose estimation takes over and finds the body, which works even when someone has their back to the camera.
The third pass is the decision. The segment gets one anchor: the settled position of the chosen subject through that shot, taken as a median so that a couple of bad frames cannot drag the framing sideways. That anchor holds for the length of the shot, and a new one is computed after each camera change. It is a deliberately conservative design — a crop that is confidently wrong for two seconds is far less damaging than one that never stops moving.
Plenty of auto-reframe implementations pan smoothly to follow a head. It demonstrates well and it is unpleasant to watch for sixty seconds. The reason is that nothing in the original footage moved — the camera was locked off — so the motion the viewer perceives has no cause in the scene. Their eye keeps trying to attribute it to something and cannot.
Holding a fixed position within a shot avoids that entirely, at the cost of occasionally framing someone slightly off centre when they lean out of position. That is the right trade. A viewer will not notice a subject sitting a little left of centre. They will absolutely notice the frame breathing.
The corollary is that the crop must move decisively when the camera does. A hard snap at a genuine cut reads as an edit, because it is one. A slow glide across the same cut reads as a software artefact. That is why detecting the cut accurately matters so much, and why subject data is used to find switches that scene detection alone would miss on a same-room multicam setup.
Fast camera movement is the hardest case. Handheld footage that whips between subjects gives the detector little to settle on, and any crop derived from it inherits the instability. If you shot handheld and the result matters, a manual pass in an editor will beat automation.
Heavy occlusion is the second. Masks, helmets, hands over the face, extreme backlight where the subject is a silhouette — detection degrades, pose fallback picks up some of it, and some footage simply will not track. The engine will still produce a sensible crop, but it is working from less information than usual.
The third case is footage where the subject is not a person at all. A whiteboard walkthrough, a product close-up, a chart, a landscape: there is no face to follow and following one would be wrong anyway. Screenshare and fit layouts exist for some of this, but the honest position is that some 16:9 content should stay 16:9. Deciding that is your call and it is a reasonable one to make.
One presenter, centred, in front of a camera that never moves. Tracking has nothing to contribute to that shot: the anchor it computes lands where a fixed crop would have landed, and you have run a detection stack to arrive at the obvious answer. If that describes your whole library, judge this on the moment selection and the captions rather than on the framing, because the framing is not doing any work for you.
It is the wrong call whenever the meaning of the shot sits somewhere other than a face. A cooking demonstration is about the hands, a repair video is about the part, a chart walkthrough is about the chart. The crop will dutifully follow the person and discard the thing the clip existed for, and there is no signal anywhere in the footage that would tell it otherwise.
It is the wrong call on material that has already been cropped once. A 9:16 file cut out of a wide original has no spare width left, so even a perfectly correct decision about who is speaking has nowhere to move the window to. Go back and feed in the widest master you still hold rather than the version you already published.
And it is the wrong call when you have a framing constraint you can state out loud — a brand mark that has to stay in shot, a lower third baked into the original, a composition built so two people sit at specific points in the frame. The engine optimises for the face and does not know your constraint exists. On those clips, set the crop yourself.
The gap between a hand-keyframed crop and an automatic one is not precision. Both can put a face in the middle of a rectangle. The gap is anticipation: an editor moves the frame a beat before the subject does, because they have already watched the shot and know what is coming. The engine only ever knows what has happened so far inside the segment it is examining.
That is the real reason the automatic path is conservative rather than smooth. A system that has no knowledge of the rest of the shot and still tries to follow motion is permanently reacting late, and late motion is exactly the drifting wobble people recognise from bad auto-reframe. Taking one settled anchor per shot sidesteps the problem by declining to play that game.
So by hand wins outright on any shot with intentional movement in it: a walk-and-talk, a reveal, somebody standing up mid-answer. Those are shots where being early is the whole craft. Automatic wins on everything else, and everything else is most of what a podcast, a webinar, a lecture or a stream actually consists of.
One asymmetry is worth knowing while you decide. A wrong automatic decision is cheap to reverse, because re-rendering that clip into a different layout takes seconds and the analysis is already cached. A wrong manual decision costs you the keyframes you set before you noticed. That, more than any argument about quality, is why the sensible default is automatic first and hand correction second.
The frame stayed on the wrong person for a whole segment. That is almost always speaker attribution rather than detection — the audio in that stretch did not clearly belong to one person, which happens with crosstalk, with laughter over an answer, or with a guest whose voice is bleeding into the host's microphone at similar level. The practical fix is to stop asking for a choice: re-render that clip as split-screen and keep both faces.
The right person is framed, but the head is sitting low or the top of it is clipped. That is the pose fallback doing its job in a segment where faces were not reliably visible, and it frames a body rather than a head. Nudging the in-point so the clip opens on a stretch where the face is clearly visible usually resolves it, because the anchor is computed from what the engine can see inside the segment you selected.
The crop repositions in the middle of what looks like one continuous shot. That is a false cut. A hard lighting change — a slide advancing behind the speaker, a lamp switching on, a big colour shift on a stream overlay — can register as a camera change, and the framing dutifully re-anchors. Trimming the clip so it sits inside a single genuine shot removes the movement entirely.
Everything is framed correctly and the whole thing just looks soft. Check the source resolution before blaming the tracking, because this one is arithmetic. A full-height 9:16 window out of 1080p leaves you around 600 pixels of width; out of 720p it leaves around 400, which then gets scaled up to fill a phone screen. No tracking decision recovers detail that the crop never had.
Frame slightly tighter than feels natural for landscape. A subject who fills a decent portion of the height gives the detector a large, stable target and gives the vertical crop room to breathe. A speaker who is small in a wide room is the hardest input there is, and it is a problem created entirely at the camera.
Light the face rather than the room. Detection depends on facial structure being visible, so a strong backlight with the subject in silhouette is genuinely difficult, whereas ordinary soft light from the front is easy. This is the cheapest single improvement available to most setups.
Keep the camera still. A locked-off shot lets the crop stay locked-off too, which is what you want. Slow drifting handheld motion translates into unstable detections and forces the framing to work harder for no gain.
If you record two people, give each of them their own third of the frame and leave the middle to the table. That is good landscape composition anyway, and it is what makes split-screen extraction clean rather than approximate. Then check one clip on a phone before you commit to a format — the difference between a crop that reads well at that size and one that does not is obvious in five seconds and invisible on a monitor.
Reviewing framing and reviewing tracking are two different jobs, and running them together is why people watch clips end to end and still miss the fault. Tracking failures live at two specific timestamps: the frame where a new shot begins, and the frame where the person talking changes. Everything between those points is a fixed anchor holding still, and there is nothing there that can go wrong.
So the process is short. For each clip, jump to every camera change and watch two seconds either side of it. Correct behaviour is a hard reposition landing exactly on the cut. A reposition that arrives a beat late, or that slides instead of snapping, means the cut was detected in the wrong place — and the fix for that is a trim, not a different layout.
Then jump to the first speaker handover in the clip, which is the most informative two seconds in the entire batch. If the frame stays on the previous person through a full sentence of the new one, the source has microphone bleed and every clip from that recording shares it. That is a session-level decision, not a clip-level one: run the whole recording in split-screen and stop looking.
Doing it in that order is the point. Tracking faults come from the recording rather than from the individual clip, so once you have identified one on the first two clips of a batch you can apply the same override to the rest without opening them. Reviewing clip by clip hides that entirely, because each failure looks like an isolated accident when it is really a property of the room.
A fixed crop you choose yourself, once per clip. Any editor will let you park a static window wherever you like in under a minute, and on a locked-off single-speaker shot that is all the framing anyone needs. It stops working the moment the recording cuts to a second angle where the subject sits somewhere else in the frame.
Running a second camera in portrait. This removes the problem rather than solving it, and for anything you record on a repeating schedule it yields a better picture than any crop out of a wide master can. It is the most valuable option here and the least often taken, because it changes the recording day instead of changing the software.
Padding the wide frame into a tall canvas keeps every pixel of the original composition and surrenders most of the screen to do it. That is the right answer for a shot whose width is the content — a stage, a pitch, a landscape — and the wrong answer for a conversation, where the reward for preserving the composition is watching two small people either side of an empty table.
Then there are the reframe features built into Premiere, Final Cut and CapCut, which all track a subject perfectly competently. Two behaviours are worth testing them on: whether they resolve one position per shot or pan continuously, and whether they will pick the person speaking rather than the largest or most central face. Those two account for most of the visible difference between any two automatic reframes, and the AI reframe tool page walks through the layout half of the same decision.
Observations from operating the pipeline in production — not general advice.
The worst reframe failure we have hit did not look like a failure. Face detection quietly stopped returning results, and every clip fell back to a static centre crop: right duration, valid file, nothing unusual on the surface. It went unnoticed far longer than a crash would have, because a badly framed video is still a video. What we took from it is that a reframe pipeline needs a positive check — assert that known-good footage still yields faces — since a run with no tracking and a run with nothing to track produce output that looks identical from the outside.
Each shot gets one crop position taken as the median of the subject's sampled positions rather than the average, and that is a response to a specific artefact rather than a stylistic preference. A single frame where the detector latches onto a face on a poster, a reflection in a window, or somebody crossing the back of the room is enough to drag an averaged position several hundred pixels sideways. Since a full-height 9:16 window out of 1080p is only about 600 pixels across, several hundred pixels is the whole distance between a centred face and an ear. A median throws that frame away.
A clip cut from a running broadcast has to be framed inside a latency budget that an uploaded file does not have, so the live path commits to a crop after sampling much less of the segment. In practice the difference is visible: the recorded path sits completely still on a seated subject, while the live path can show small residual movement on the same kind of shot. We treat the two as separate engineering problems for that reason, and we would rather write that down than let anyone assume a mid-stream clip and an uploaded clip are framed by identical logic.
A skilled editor with position keyframes will beat this on any single clip, especially difficult handheld material. It is also perhaps ten minutes per clip, on top of finding the moment and captioning it. The comparison that matters is not one clip against one clip, it is whether you produce fifteen or two.
Those features work and they are convenient if you already have the project open. They generally track a subject continuously rather than resolving one position per shot, which produces the drifting motion described above, and they will not choose the speaker on a two-person set. They are also a step inside an edit, whereas this arrives with the clip already found and captioned.
Faster than anything and correct on exactly one kind of footage: a single person, deliberately centred, with a camera that never moves. If that is genuinely what you shot, a centre crop is fine and you do not need tracking. Everything else is where the difference shows.
Padding the landscape frame into a vertical canvas keeps the whole picture and gives up the screen. On a phone the subject ends up postage-stamp sized in a band across the middle, with the rest of the display doing nothing. It preserves the composition at the cost of the thing composition was for. Try a reframe on your own footage and compare the two side by side.
Run it on a shot where someone stands up and walks, then keep your eye on the edge of the crop rather than the person inside it. Drift or snap — that one behaviour answers the whole question in about five seconds.
⚡ Reframe My Video — $1 trial
