FastClip partial file restore uses a real time index system to quickly produce small, pointer files that track the actual media within almost any broadcast or post media file. These files are small enough to keep on fast, local or network storage, and alleviates the need to look at the source file tables/metadata/extra data. When the part of the file that needs to be restored is determined, from time code or absolute time code, the index file can be used to read only the parts of the file needed to create the new partial file.

Compatibility:
	Windows, Windows Server, Red Hat/CentOS or Ubuntu OS

Net-X-Player

Net-X-Code's FastClip partial file restore provides a complete, index/proxy/metadata/restore capability for Windows, Windows Server, Red Hat/CentOS or Ubuntu OS.

Source and Proxy File Support

Partial File Restore provides support for a wide range of industry standard files, including MXF, GXF, LXF, MOV, AVI, DPX, CINE, RMF, VFW, and CDX. For a full list, please see file format support in videoQC. Along with indexing, the system can produce proxy files with matching metadata and time code.  The proxy files can be read with standard preview players such as QuickTime, Windows Media Player, VLC, videoQC, and/or viewed with time code across a custom HTML5 web player. The Proxy files are time code accurate and can be used to create edit decision lists to edit the main (high resolution) source files.

Google/Amazon/Cloud, SMB, NFS, NAS, HTTP, FTP and SAN Support

The offline/nearline source files can be accessed via a number of methods.  One of the main access methods is via the cloud, including FTP and HTTP direct access.  Shared file systems like SMB/CIFS, mapped HTTP and NFS files are also accessed directly.  SAN or attached storage can be any OS available drive.  If using nearline or cloud based storage, the index files are optional.  They significantly speed up initial open, but restore speed is equally fast with or without them.

HTTP/RESTful API, Single or Multi Server Configs

Restore groups can be contained on one or more servers.  Each daemon process can run independently or as a group on a server.  The central server process exports an HTTP RESTful/AJAX interface that can be used from within a web browser, or be controlled by user API code in any programming language that supports HTTP get and XML.

FastClip

FastClip is a system in Partial File Restore and Net-X-Code that allows clipping of files and live streams for direct output to web servers.  This allows extremely fast turnaround of video assets to external servers or even full resolution assets for immediate use in editing or on-air run down. Because everything is based on time code and index files, different level/bitrate files can be clipped simultaneously to create multi bitrate video for multiple devices and bandwidth requirements.

Extra Functions

To make the system more complete, a number of accessory functions are also provided:

• Thumbnail generation
• Metadata extraction
• Closed Caption extraction
• Wave/Audio extraction

Workflow Overview

The partial file restore begins with a source file.  Almost any broadcast, post production or web based file is supported.  An RTIndex is created from the original file.  This small pointer file is very quick to create and allows fast direct access to the media in the source file.  This file can be stored on a local drive, NAS or on the tape/cloud.  A secondary process can, at the same time, create a frame for frame duplicate proxy file with matching metadata and time code. These files are normally moved to a web facing database server. The file can be played back via HTML5 video with time code, or with many other players. The primary file is then moved off to tape, NAS, cloud, AWS or other long term storage. When a section of a file is needed, the proxy file can be used to determine the in and out time code of the segment. These time code points are then used to reference the original file. That section of the file can then be used to create a new file, either with no conversion (re-wrap), or converting it to another standard format. Only the minimum data required is actually read from the source file.

Typical Proxy, Convert and PFR Session

This section has some typical proxy, convert and partial file restore workflows. The first scenario is a complete list. The other scenarios assume the first few steps in the first scenario have already been accomplished.

Scenario 1 – full access

• The file (MXF, MOV, AVI, CINE, etc.) arrives at ingest
• A “command=copy&profile=index” command is sent to index the original file name
• A “command=copy&profile=mp4-h264” command is sent to make a proxy file with time code, multitrack audio, closed captions, metadata and proxy index file name
• Two “command=copy&pisrc&pidst” commands are sent to create JPEG images for the source and proxy files
• At this point, the proxy and main index can be stored in a real or near real time storage, and the main file may be moved to long term storage, tape, cloud (google/s3) or other offline storage
• The HTML5 player’s time code (or other time code source) is then used to set one or more In and Out points on the file that needs to be restored.
• A “command=copy&profile=wrap” is sent to access the bytes of the original file and create a new file of the same type, without any recompression of audio or video, at the target location.

Scenario 2 – tape restore

• This assumes the basic processing in Scenario 1 has been done
• A “command=getcopyinout” is sent with the absolute or time code based in and out points, and the index of the source file it will come from
• This returns a series of one or more file names with start and end byte locations
• At this point, the controller restores those byte areas of the files to the name specified by the return
• Once the areas are restored, a “command=copy&profile=wrap” is sent along with the temp folder to create the new output filename

Scenario 3 – cloud restore

• This assumes the basic processing in Scenario 1 has been done
• If the index file is stored on cloud, it can be restored locally first, or read directly from the cloud (HTTPS, FTPS, AWS)
• If the main file is in Glacier, then a command will be sent to restore the section needed to S3 before the restore is done
• Once there is access to the file, or file part, the “command=copy&profile=wrap” can be called normally
• If the resource is in available cloud storage (e.g. not Glacier), then partial file restores may be done from the original file without indexing it first. This will cause more data to be read, but only the headers and tables necessary to find the audio/video/data the restoration needs.

Scenario 4 – in line conversions

• This assumes the basic processing in Scenario 1 has been done
• For any restore scenario, the file restored can be a byte accurate re-wrap of the original into a new container, or it can be translated in the process (on the fly) to any supported standard format. These formats include MXF Op1a, Op-Atom, P2, IMX, D11, IMF, MOV, Uncompressed and many other containers, with codecs including JPEG-2000, XDCam, MPEG-2, h.264, HEVC, AVCi 100/200, XAVC-S, XAVC, Long-G, TR-01, DV and many others.
• The commands can also be used with or without index files to convert all or part of local clips to any of these formats.

 Partial List of Supported Formats

• AVI - Uncompressed, h.264, MJPEG, Uncompressed, MPEG-2, DVHD/25/50
• Avid DS Gen - Uncompressed
• Cine - Raw bayer
• DNG/KRW/VRW/CDX - Raw bayer sequence of stills
• DPX/CIN/HDR/SGI/TIFF/PNG/PSD/TGA/BMP - Image sequences
• DVS - Uncompressed RGB or YCbCr
• GVK2 - Grass Valley K2 file groups
• GXF - SMPTE 360
• IHSS - Speed grade uncompressed
• KRW - Kinefinity raw bayer
• LXF - Harris MPEG-2, DVHD/25/50, AVCi-100
• MLV - Magic Lantern raw bayer
• MOV - ProRes, DVHD/25/50, AVCi, h.264, HEVC, most QuickTime codecs
• MP4 - h.264 (4:2:0, 4:2:2, 8/10/12 bit), HEVC  (4:2:0, 4:2:2, 8/10/12 bit), Sony XDCam
• MTS - h.264 camera captures
• MXF - OP1a, OP1b, OP-Atom, EVS, Uncompressed, AVC, XAVC, AVCi, DVHD/25/50, XDCam,
• MPEG-2, IMX, h.264, HEVC, Sony Raw, HDCam, F5/F55, Long-G, etc.
• R3D - RED camera files
• SEQ/SIV - Raw bayer camera files
• TR-01 - JPEG-2000 TS-01/Evertz transport streams
• TS - MPEG-1, MPEG-2, JPEG-2000, h.264, HEVC transport streams
• VCAP - Tektronix raw files
• YUV/RAW - raw uncompressed data

Please contact us for the most up to date list.

Net-X-Code API (includes PFR, Net-X-Convert, Net-X-Copy)

Net-X-Code API (pdf)

videoQC is a suite of video and audio analysis and playback tools with both visual and automated quality checking tools. videoQC will take the media coming into your facility and perform a series of automated tests on video, audio and metadata values against a template, then analyze the audio and video. It includes...

Video I/O Software

Net-X-Code is a group of IP capture, control, convert and output applications designed to run on one or more servers on your LAN, in the cloud, or as a hybrid....

Network Streaming/PFR

FlowCaster lets you share high quality video over IP for remote collaboration and review workflows. Securely share the output of your Adobe/Avid/Resolve etc. editor, with time code, closed captions, metadata, at extremely low latency...

Network Streaming/PFR