Workflow:Watch Folder Ingest

Watch Folder Ingest
Status:	Production
Tools:	Python OVP MAM DAM API OTT
Input:	Media assets and descriptive metadata in Watch Folder.
Output:	Media assets and descriptive metadata in destination OVP.
Organisation:	La Digitalizadora de la Memoria Colectiva

Workflow Description

Watch Folder Sequence Diagram UML

                             ┌──────┐                                              ┌──────┐          ┌───┐                    
                             │Script│                                              │Folder│          │OVP│                    
                             └──┬───┘                                              └──┬───┘          └─┬─┘                    
                               ┌┴┐              Start monitoring folder               │                │                      
                               │ │ ──────────────────────────────────────────────────>│                │                      
                               │ │                                                    │                │                      
                               │ │                                                    │                │                      
         ╔═══════╤═════════════╪═╪════════════════════════════════════════════════════╪════════════════╪═════════════════════╗
         ║ LOOP  │  Check for new files                                               │                │                     ║
         ╟───────┘             │ │                                                    │                │                     ║
         ║                     │ │                 New file detected                  │                │                     ║
         ║                     │ │ <──────────────────────────────────────────────────│                │                     ║
         ║                     │ │                                                    │                │                     ║
         ║                     │ │                                                    │                │                     ║
         ║         ╔══════╤════╪═╪════════════════════════════════════════════════════╪═════════════╗  │                     ║
         ║         ║ ALT  │  File is MP4 video                                        │             ║  │                     ║
         ║         ╟──────┘    │ │                                                    │             ║  │                     ║
         ║         ║           │ │────┐                                               │             ║  │                     ║
         ║         ║           │ │    │ Check for corresponding XML file              │             ║  │                     ║
         ║         ║           │ │<───┘                                               │             ║  │                     ║
         ║         ║           │ │                                                    │             ║  │                     ║
         ║         ║           │ │                   XML file found                   │             ║  │                     ║
         ║         ║           │ │ <──────────────────────────────────────────────────│             ║  │                     ║
         ║         ║           │ │                                                    │             ║  │                     ║
         ║         ║           │ │────┐                                               │             ║  │                     ║
         ║         ║           │ │    │ Extract metadata (title, description, tags)   │             ║  │                     ║
         ║         ║           │ │<───┘                                               │             ║  │                     ║
         ║         ╠═══════════╪═╪════════════════════════════════════════════════════╪═════════════╣  │                     ║
         ║         ║ [File is XML]                                                    │             ║  │                     ║
         ║         ║           │ │────┐                                               │             ║  │                     ║
         ║         ║           │ │    │ Check for corresponding MP4 file              │             ║  │                     ║
         ║         ║           │ │<───┘                                               │             ║  │                     ║
         ║         ║           │ │                                                    │             ║  │                     ║
         ║         ║           │ │                   MP4 file found                   │             ║  │                     ║
         ║         ║           │ │ <──────────────────────────────────────────────────│             ║  │                     ║
         ║         ║           │ │                                                    │             ║  │                     ║
         ║         ║           │ │────┐                                               │             ║  │                     ║
         ║         ║           │ │    │ Associate metadata (title, description, tags) │             ║  │                     ║
         ║         ║           │ │<───┘                                               │             ║  │                     ║
         ║         ╚═══════════╪═╪════════════════════════════════════════════════════╪═════════════╝  │                     ║
         ║                     │ │                                                    │                │                     ║
         ║                     │ │────┐                                               │                │                     ║
         ║                     │ │    │ Check if file pair has already been processed │                │                     ║
         ║                     │ │<───┘                                               │                │                     ║
         ║                     │ │                                                    │                │                     ║
         ║                     │ │                                                    │                │                     ║
         ║         ╔══════╤════╪═╪════════════════════════════════════════════════════╪════════════════╪═══════════╗         ║
         ║         ║ ALT  │  File pair not processed                                  │                │           ║         ║
         ║         ╟──────┘    │ │                                                    │                │           ║         ║
         ║         ║           │ │                  Upload video to OVP with metadata │                │           ║         ║
         ║         ║           │ │ ───────────────────────────────────────────────────────────────────>│           ║         ║
         ║         ║           │ │                                                    │                │           ║         ║
         ║         ║           │ │                          Upload successful         │                │           ║         ║
         ║         ║           │ │ <─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─│           ║         ║
         ║         ║           │ │                                                    │                │           ║         ║
         ║         ║           │ │────┐                                               │                │           ║         ║
         ║         ║           │ │    │ Add file pair to processed list               │                │           ║         ║
         ║         ║           │ │<───┘                                               │                │           ║         ║
         ║         ╠═══════════╪═╪════════════════════════════════════════════════════╪════════════════╪═══════════╣         ║
         ║         ║ [File pair already processed]                                    │                │           ║         ║
         ║         ║           │ │────┐                                               │                │           ║         ║
         ║         ║           │ │    │ Skip processing                               │                │           ║         ║
         ║         ║           │ │<───┘                                               │                │           ║         ║
         ║         ╚═══════════╪═╪════════════════════════════════════════════════════╪════════════════╪═══════════╝         ║
         ╚═════════════════════╪═╪════════════════════════════════════════════════════╪════════════════╪═════════════════════╝
                             ┌─└┬┘──┐                                              ┌──┴───┐          ┌─┴─┐                    
                             │Script│                                              │Folder│          │OVP│                    
                             └──────┘                                              └──────┘          └───┘                    

I want to monitor a local folder for new content. When a new file becomes available, the system will check if it's either an MP4 video file or an XML file. If it's an MP4 video, the system will then search for a corresponding XML file with the same name. Upon finding the XML file, it will extract metadata such as title, description, and tags. Subsequently, the system will upload the video file to an Online Video Platform (OVP) along with the extracted metadata. To prevent duplicates, before processing a new file pair (MP4 and XML), it will verify whether the pair has already been processed.

Purpose, Context and Content

Purpose: A watch folder serves as a designated directory that is monitored by a specific application or process. The primary purpose of a watch folder is to automate tasks based on the contents of that folder, facilitating Automated File Processing, Workflow Automation and Data Ingestion.

Context: In the context of digital content migration, this blueprint serves as a guide for facilitating bulk media and metadata ingests.