20

I'm a software developer and I'm also interested in photography (for four years) and video production (for a few months only).

■ In software development, there is an important rule every developer follows on every project: everything should be under version control: source code, configuration files, database schema, documentation—everything which enables building the project from scratch. This has two pleasant consequences:

  1. In an event of a disaster when you lose everything except the version control repository, you should be able to continue as if nothing happened.

  2. In an event of some stupid change which negatively affects the project, the developer can come back to an earlier revision.

■ In photography, every change I make to photos is stored forever in Lightroom catalog, making it possible to revert to previous state at any moment. With virtual copies feature, Lightroom also enables to do what is called a branch in version control: the ability to test something different, and either keep both results or remove one of them later.

The catalog doesn't store the RAW photos themselves, but they don't change anyway.

■ In video production, things seem different. I work with Premiere Pro, After Effects and Soundbooth.

  • None seem to store history permanently: if I do an action by mistake and notice it only the next day, there is no way to recover the previous version.

  • Soundbooth also changes directly the WAV files, which requires an additional effort to keep separate the original recordings from the modified ones.

  • Version control is rarely mentioned, and I haven't found anyone telling how does he actually use version control in his workflow. Moreover, nobody mentions which version control should be used, and since most version control systems are optimized for text files, not binary ones, this creates an additional challenge.

  • Video.SE doesn't have or tags.

Thus, I have two questions:

  1. Does version control have a part in the workflow of a person who works with video production? How is it integrated?

  2. Would migrating to Adobe Creative Cloud help? Are there specific features which allow, in Creative Cloud, to track successive revisions of a Premiere Pro or After Effects project?

Remark: to avoid off-topic answers, I highlight that my question is unrelated to the backups, and is specifically about storing successive revisions of my work, not having an on-site/off-site backup of the data.

10

Version control in the sense of Git isn't very practical in the video world. You would need to make a specific version control tool for every audio and video tool out there as all work with their own project formats. But being able to read these formats is just one thing, then you also need the render engine of that tool to show diff's.

Though all of these tools work in a non-destructive way unless you pre-render some stuff (compare that to compiling a dll/lib of a piece of your code and work with that from now on), so you ususally can just go back to an old revision by doing ctrl+z or using the history tool in some programs.

Though saving sub versions is usually the way to go. Like stib described in his answer or by doing it manually.

Something I like to do and works well with every software is putting my project files (no source footage) into Dropbox. If you have a somewhat fast upload speed (~1 Mbit/s) and your project file isn't 100MB+, you can upload your project before you save the next time. An average Premiere/AE/FCP project is around 10-20MB so your recently saved files will be uploaded in 1-2 minutes. Even faster if you have more upload bandwidth.

Then if you have to go back you can access Dropbox's history of your files and download or restore to that revision. Dropbox saves file revisions forever* on a paid account (*atleast when they had the pack rat option, now its a year I think) and for 30 days on a free account. I'm sure that there are other cloud hosters that offer similar features. It's a bit like using a super limited version of git that handles binary files very well and is headache free. This has the advantage that you don't clutter the folder with tons of files and have a backup at the same time.

Most cloud hosters also offer Team Memberships so you can work with multiple editors. Or you share the project folder with other team members.

  • Dropbox stores file revisions forever? Then what if you had a 5 GB video file with 800 revisions? – Pacerier Oct 22 '14 at 7:08
  • I edited the answer a bit. With the old pack rat option it was indeed forever now its a year, and yes you can do what you just described but like I wrote in my answer don't put source (e.g. video/image/audio) files in there, only your project files. Unless you have a gigabit connection that would be very unpractical to keep versions of changing source or render files. – PTS Oct 22 '14 at 13:59
  • 1
    You only need your VCS to understand the formats if you need diffs. That's a bit much to expect. – Peter Cordes Mar 7 '15 at 21:39
  • 1
    So you could easily version control your project files, since 10-20MB of binary data is no problem for git. You just need to write useful commit messages to describe what state your save file was in when you commited it. If you're lucky, then small edit changes often don't change most of the bits in the project file, and git's delta compression will use a lot less than the full 10MB for every commit. And then backups are as easy as git push to your backup server. (with some other method for keeping track of which video masters go with which project, maybe md5sums of the source files?) – Peter Cordes Mar 7 '15 at 21:43
  • Well thats the ideal case, if your project ends up with a larger project file you just can't upload revisions as often depending on your connection. A cloud storage software scales perfectly, git will eventually run into troubles. Commit messages are definitely a plus for git though. – PTS Mar 7 '15 at 22:31
6

Just to add to the previous responses: While there's nothing quite like Git for the video world, there are Digital Asset Management/Media Asset Management tools that can more or less do the same thing - version control and permission/user management (they also do a lot more, as they're really built as libraries for your media). For years, I used Apple's Final Cut Server app (now obsolete) that integrated with the Final Cut Suite (Final Cut Pro 7, Soundtrack Pro, etc) in a small post facility.

We used it for version control and branching on project files, which allowed multiple editors to work on a single project relatively seamlessly. Because this was an Apple product, it was designed to be used with Final Cut Pro and therefore could read and work with FCP project files very easily. Even given this, Final Cut Server's version control relied on saving previous versions of the entire project file, it didn't use diffs. I don't know of any DAM that does for the very reason that a previous answer already pointed out - there are way too many proprietary formats (even though, ironically, many of them now rely on XML as the backbone for those project file formats).

FCS was great because it was relatively affordable. There was never really anything analogous for Premiere Pro. Nowadays, unfortunately, you'll need to hand out a nice chunk of change to get similar capabilities - mostly because these tools are really designed for post facilities, not a single editor. They also require potentially significant integration/setup.

Here are a few options (I have no relationships with any of these companies, this is based purely on my research looking for a similar solution):

5

Version control doesn't really have as much of a place in video editing because it is by nature non-destructive. At the core of any NLE(non-linear video editor), the output is actually something known as an Editing Decision List or EDL. This is extremely analogous to the history in Lightroom as that history is a record of all changes that have been applied in order.

NLEs work from source clips. They take beginning and end points of those clips to place them into a timeline and then effects can be applied to those assets in a given order (based on placement of the effects), however these are all editing decisions and are applied on the fly (or possibly rendered out to temporary preview files). The final output render is the result of applying the entire EDL to the source clips.

You could save a version of the project in order to be able to go back to a previous version of the EDL if you want, but this isn't generally needed unless you are very intentionally branching to try an alternate approach to editing a sequence (in which case a copy of that timeline is often a better choice anyway.)

  • What do you mean by "non-destructive" and NLE here? – Pacerier Oct 22 '14 at 7:09
  • An NLE is a non-linear editor (technical name for most video editing software.) Non-destructive means that changes don't result in destruction of assets. It is forming a list of changes to make, which is basically the same thing that version control makes. When you make changes to code, that is destructive because your new alterations destroy your old. With an NLE, your assets all remain unaltered and you are just modifying a list of sections to use and filters to apply. – AJ Henderson Oct 22 '14 at 13:30
  • So do you mean that we can tell the video editing program "revert to version 2.5", edit abit, then tell it "revert to version 7" and it is able to do that? – Pacerier Oct 22 '14 at 21:45
  • 1
    No, he means you always still have your master video. The assumption is that reproducing the effects/cuts you had earlier isn't hard, or that saving projects files with a VCS would be more work than just re-inputting the edit decisions. If that's not the case, then sure, version control your project files. Although without the ability to merge changes from different branches, it's probably more useful to just write down any exact frame numbers that it took you a long time to find out was the right place for a cut, or something. – Peter Cordes Mar 7 '15 at 21:47
3

If you enable it in the preferences After Effects and Premiere automatically make incremental saves of the project files. autosave preferences

These incremental saves could be used to revert to previous versions, which is like a very basic implementation of version control (you might want to increase the number of versions from 5 though). FCP has a "restore from previous version" function built in, which is good for when it mangles your project files. After effects has (but Premiere doesn't have, go figure) the ability to incrementally save a project. I use this all the time when I'm making big changes to a project, and want to be able to revert to the main trunk, so to speak.

For added control, you could I imagine, use version control software to manage the folders where you keep your project files and auto-saves, so editors would checkout the current cut and commit changes, as long as all the media was centrally accessible or copied onto everyone's machines in the same relative path. It wouldn't let you fork and merge other people's edits, like you can with code – that would be an interesting feature (I'd say it could be implemented with Adobe's extendscript scripting, as long as your skills are up to re-writing Git or SVN in Javascript).

  • 1
    yeah, to diff and merge, either your VCS has to understand your NLE's save files, or the NLE has to provide diff / merge. (e.g. Given a program that can merge changes in a project file, given a common ancestor, I think you could set up git to use it for git mergetool, to be able to merge commits of trees containing modified project files.) – Peter Cordes Mar 7 '15 at 21:52
  • You could possibly use extendscript to 'save' your current project by getting all the properties of all your project items and saving the data to a file. But for even a moderately large project that might take a long time to do. If you did that you could possibly build a version control system. – stib Jan 4 at 1:40
3

As a long-term video professional i can attest to the fact that the need for a lightweight, robust, transparent and open form of a VCS is sorely lacking in the majority of media workflows. The problem however is multi-faceted and is a cultural one as well as a technical one.

Traditionally we have been working in a sausage factory like manner where a project is greenlit from script, moves to production, once wrapped it goes to post-production and then a final output is delivered to the distribution arm who then spin off device/platform outputs.

Nowadays this factory-like approach is an illusion where the hand off between post-production and distribution specifically is never clear. There is a lot of back and forth with cuts/edits for different languages/markets nevermind going back and doing re-mastering for the latest format for example. Then there is the need to access final versions for marketing purposes... As a result the need for not only remote parties but people who may never meet to have a catalogued, definitive understanding of what version they need to work on is key. This extends not only to master encodes but all versions of that master for different markets as well as the versions of the assets that were used to create each master.

Only now are the media tech community engaging on what a version truly is and it is regularly debated due to the myriad of different workflows and concerns. I break it down as a working version and a distribution version. There are efforts to remedy this within distribution by creating an archive file format that tracks versions within itself (to combat the fact that there are multiple tools, platforms etc) - this is called the Interoperable Master Format (IMF - not to be confused with the bank) and is being steered through SMPTE. The good thing about this is that it is moving to provide interoperability between the myriad of digital asset management systems (those that want to support it) that are out there - some studios i know have asset management systems that number in the hundreds - this would help them internally let alone for external handoffs. Of course it has not been used within a production environment yet seeing as it is designed as an archive level format (Netflix now utilise it). It is also a very hefty file with no easy way of creating it unless you have the necessary capital to invest in the tools. Netflix did release an open source tool set that provides reading ability which is nice.

Working version or production level wise i feel that there is a need to provide a VCS (such as a modified form of git perhaps) that anyone can utilise no matter how large or small in order to facilitate remote working. Media files are of course a lot larger than swapping code or libraries but the decisions made on those files are the key component. I for one would like to test working remotely via git commits if only to avoid the naming conventions of 'file_Final_FINAL_MASTER_version3.mxf' naming conventions that get swapped back and forth.

2

I had the same question, also being a software engineer by trade, thinking of photoshop work.

we can tell the video editing program "revert to version 2.5", edit a bit, then tell it "revert to version 7" and it is able to do that?

I found that Photoshop it lets me set a named version in the history, and I think that's saved in the file...? For revisions (entries in the history list) that are not named, the nodes are lost from the display when an edit is made to a previous place (branching) and there is no reflog exposed.

New versions of Premiere seem to have a similar history log and I'm guessing is evolving toward the same internal architecture, where each change is another copy of the project that shares most of the statenwithmthe previous. If the history has checkpoints that are saved, it is very much like a git store: each version contains (shared) references to underlying elements on down to the Segment definitions. As the video itself is not in the file, it lends itself well to growing more and more versions with little increase in size.

I saw a seminar where someone on the Photoshop development team explained the architecture. It looks like the history entries you see are analogous to git versions, like gitk displays. Naming the version is the same as a git tag. You can reset to any visible revision by pointing to it, and reset back too. But making any change that is itself put in history is like doing a full refresh (shift or ctrl F5)— you lose whatever is not chained down to from the current branch head or named tag (but I think things like clone-source references still point to the now-unseen version).

But that's not what I'm writing to suggest. I set the NAS volume where my project resides to make a snapshot every 3 hours. Windows has a checkpoint mechanism but I think it's not configurable; Mac Time Machine does something similar.

In general, you can archive all saved versions of the file, and in Premiere that is not containing all the imported (constant) assets, so it is reasonable to save them all, even without being able to use deltas to save only what changed.

Just re-learning Premiere and getting more agressive with trying things, I'm confidant that I can revert if next time I work on it I regret what I did, or found a better way and want to do it again. That is an effective revision versioning system. Doing it on the NAS, I'm also protected against a BSOD trashing the entire project when saving. :)

update the history is a short length, defaults to 32 entries. It is empty when the project is loaded. However, the auto-save doesn't just save over the same file like we see in most programs; rather it numbers them and keeps them. So, I can see the file timestamps and load an older copy, which gives me a version history of 15-minute checkpoints. In my case, each file is 44K, which is nothing compared to the asset size— it's the size of 76 milliseconds of audio, or 1/7th of a frame of class 10 SD card footage.

If you have the presence on mind to keep a checkpoint with a meaningful name, just Save copy As. But the autosave, set to a high frequency, can be used to revisit any state (to that time granulatity) without advance planning, with little effort.

A note to those non-engineers who are not familiar with version control: Besides the ability to backtrack your work in the obvious way, I also often use it to check what I just changed, or compare with the state of before beginning the current task, or compare with the last verson that's been shared with the group.

Since Premeire now supports opening multiple projects in the workspace, it would be feasible to have a window arrangement workspace setting for comparing two timelines. That is, make more effective use of having those versions, not just for backup. I often tell programmers who don't use git how it becomes a general purpose tool, like a text editor.

I wonder how professional filmmakers handle version control, if anything more than ad-hoc? The autosave design seemsmquite purposful, and the integrated script-writing groupware tool does have explicit visible revision tracking.

0

Having version control for video files themselves is impractical because first, they are huge, second they are moving (are you going to preserve every frame?), third, they are immutable. That is, the original files do not ever change when editing.

But version control for project files makes a lot of sense. Presently, after every significant change I create a new project file and give it some descriptive name - what did I do, what did I add and what did I remove. In essence, I have to maintain the history manually by means of file names. Having project files under version control is a great idea, why I haven't thought of this before!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.