Jump to content

Uploading videos to YouTube Automatically. Python Node

Miguel Angel Doncel
Miguel Angel Doncel
in Python Scripting

Featured Replies

Posted

Hi all,

We have created a Py examplenode to upload videos to youtube as part of the Mistika Workflows process. It will be included in the next Open Beta. 

Meanwhile, in order to help users/developers to use it and understand the code, let me explain the code:

 

In order to use it, you may need to install the following py modules in your system (if they are not installed already):

  • google-api-python-client
  • oauth2client
  • httplib2
  • argparse

the initialization function: 

def init(self):
    self.addConnector("File",Cconnector.CONNECTOR_TYPE_INPUT,Cconnector.MODE_REQUIRED)
    self.addConnector("Output",Cconnector.CONNECTOR_TYPE_OUTPUT,Cconnector.MODE_OPTIONAL)
    self.addProperty("mail","")
    self.addProperty("title","test upload. to be deleted")
    self.addProperty("description","File uploaded with Mistika Workflows")
    self.addProperty("category",22)
    self.addProperty("keywords")
    self.addProperty("privacyStatus","public")    
    self.addProperty("loginTimeout",30)    
    return True

- The Connectors:

The node has 2 connectors:

"File":  the input movie(s). Their format must be already compatible with youtube. In order to do that, you can use any previous Transcoding node.

"Output": The output connector will include the input universal Paths including a new parameter with the video youtube ID. That way, it can be used in the following nodes, i.e. to send a link by mail, or slack.

- The Properties:

The Node uses 7 properties

mail: This mail is used to store the Youtube access Token. 

title: The video title.

description: The video description.

category: The Youtube category number. The names are defined in the .xsd file, and the default (22) correspond to "People & Blogs category"

keywords: comma separated keywords,

privacyStatus: It can be Public, Private or Unlisted. Default is Public.

loginTimeout: It is a hidden parameter with the login process TimeOut. After that time, if the user does not grant access  permissions in the web Browser, the process will fail.

the validation function: 

def isReady(self):
    res=True
    if not self.mail: 
        res=self.critical("youtube:isReady:mail","email can not be Empty")    
    if not self.title: 
        res=self.critical("youtube:isReady:title","Title can not be Empty")    
    if not self.description: 
        res=self.critical("youtube:isReady:description","Description can not be Empty")
    return res

The validation funcion is very simple, it just checks that the mail, title and description files are not empty.

-The process function: 

def process(self):
    out=self.getFirstConnectorByName("Output")
    out.clearUniversalPaths()
    res=True
    success = False
    port_number = 0
    args = argparser.parse_args()
    socketError=False
    httpd=None
    timeout=self.loginTimeout
    for port in args.auth_host_port:
	port_number = port
	try:
	    httpd = ClientRedirectServer((args.auth_host_name, port),ClientRedirectHandler)
	    httpd.timeout=timeout
	except socket.error,e:
	    socketError=e
	    pass
	else:
	    success = True
	    oauth_callback = 'http://{host}:{port}/'.format(host=args.auth_host_name, port=port_number)
	    break
    if not success:
	self.critical("youtube:process","unable to start local web browser: %s" %socketError)
	if (httpd):
	  closeServer(httpd)
	return False
    print('server running on port {}'.format(httpd.server_port))
    for c in self.getConnectorsByName("File"):
        for p in c.getUniversalPaths():
            file=p.filePath()
            print "uploading %s" % file
            id=uploadFile(self,file,args,httpd,oauth_callback)
            if id:
                p.setParam("youtubeId",id)
                out.addUniversalPath(p)
            else:
                res=False
    closeServer(httpd)
    return res

The process funcion first creates a server to process  the youtube  identification callback in the local computer. That server will listen the ports 8080 or 8090 if they are not used already.

Once the server is created, the node will process all the input connectors and all their universalPaths to upload them to YouTube.

The upload Process:

The upload process is based on the youtube example available at https://developers.google.com/youtube/v3/docs/videos/insert

In that link you can find details about that part of the process.
The code is pretty much maintained from that example, we just made the necesary adjustments to make it work inside Mistika Workflows.

The only significant change was to move the server creation outside the uploadFile function to avoid creating and destroying the server for every upload,  doing it once instead.

 

So, the full node code is the following: 

import httplib
import httplib2
import os
import random
import sys
import time
import logging
import socket

from argparse import Namespace
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
from googleapiclient.http import MediaFileUpload
from oauth2client.client import flow_from_clientsecrets
from oauth2client.file import Storage
from oauth2client.tools import run_flow,argparser,ClientRedirectServer,ClientRedirectHandler
from oauth2client import client
from Mistika.classes import Cconnector

def init(self):
    self.addConnector("File",Cconnector.CONNECTOR_TYPE_INPUT,Cconnector.MODE_REQUIRED)
    self.addConnector("Output",Cconnector.CONNECTOR_TYPE_OUTPUT,Cconnector.MODE_OPTIONAL)
    self.addProperty("mail","")
    self.addProperty("title","test upload. to be deleted")
    self.addProperty("description","File uploaded with Mistika Workflows")
    self.addProperty("category",22)
    self.addProperty("keywords")
    self.addProperty("privacyStatus","public")    
    self.addProperty("loginTimeout",30)    
    return True

def isReady(self):
    res=True
    if not self.mail: 
        res=self.critical("youtube:isReady:mail","email can not be Empty")    
    if not self.title: 
        res=self.critical("youtube:isReady:title","Title can not be Empty")    
    if not self.description: 
        res=self.critical("youtube:isReady:description","Description can not be Empty")
    return res
    
def process(self):
    out=self.getFirstConnectorByName("Output")
    out.clearUniversalPaths()
    res=True
    success = False
    port_number = 0
    args = argparser.parse_args()
    socketError=False
    httpd=None
    timeout=self.loginTimeout
    for port in args.auth_host_port:
	port_number = port
	try:
	    httpd = ClientRedirectServer((args.auth_host_name, port),ClientRedirectHandler)
	    httpd.timeout=timeout
	except socket.error,e:
	    socketError=e
	    pass
	else:
	    success = True
	    oauth_callback = 'http://{host}:{port}/'.format(host=args.auth_host_name, port=port_number)
	    break
    if not success:
	self.critical("youtube:process","unable to start local web browser: %s" %socketError)
	if (httpd):
	  closeServer(httpd)
	return False
    print('server running on port {}'.format(httpd.server_port))
    for c in self.getConnectorsByName("File"):
        for p in c.getUniversalPaths():
            file=p.filePath()
            print "uploading %s" % file
            id=uploadFile(self,file,args,httpd,oauth_callback)
            if id:
                p.setParam("youtubeId",id)
                out.addUniversalPath(p)
            else:
                res=False
    closeServer(httpd)
    return res
  
# YouTube auxiliary functions. 
# Based in google example available at https://developers.google.com/youtube/v3/docs/videos/insert
def uploadFile(self,file,args,httpd,oauth_callback):  
    if not os.path.exists(file):
        self.critical("youtube:upload:fileNotFound","File Not Found: %s" % file)
        return False
    
    httplib2.RETRIES = 1
    MAX_RETRIES = 10
    VALID_PRIVACY_STATUSES = ("public", "private", "unlisted")
    setattr(args,'file',file)
    setattr(args,'title',self.title)
    setattr(args,'description',self.description)
    setattr(args,'category',self.category)
    setattr(args,'keywords',self.keywords)
    setattr(args,'privacyStatus',self.privacyStatus)
    youtube = get_authenticated_service(self,args,httpd,oauth_callback)
    if (youtube):
      try:
	  id=initialize_upload(self,youtube, args)
      except HttpError, e:
	  self.critical("youtube:upload:httpError ","An HTTP error %d occurred:\n%s" % (e.resp.status, e.content))
	  return False
      return id
    return False
    
def get_authenticated_service(self,args,httpd,oauth_callback):
  
    CLIENT_SECRETS_FILE = "{}/youtube/workflows.json".format(sgoPaths.workflowsLibrary())
    YOUTUBE_UPLOAD_SCOPE = "https://www.googleapis.com/auth/youtube.upload"
    YOUTUBE_API_SERVICE_NAME = "youtube"
    YOUTUBE_API_VERSION = "v3"
    
    flow = flow_from_clientsecrets(CLIENT_SECRETS_FILE,scope=YOUTUBE_UPLOAD_SCOPE)
    CREDENTIALS_FILE = "{}/{}".format(sgoPaths.tmp(), self.mail)
    storage = Storage(CREDENTIALS_FILE)
    credentials = storage.get()
    
    if credentials is None or credentials.invalid:
        credentials = run_touyube_flow(self,flow, storage, args,httpd,oauth_callback)

    if credentials:
      return build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, http=credentials.authorize(httplib2.Http()))
    return False

def initialize_upload(self,youtube, options):
    tags = None
    if options.keywords:
        tags = options.keywords.split(",")

    body=dict(
        snippet=dict(
            title=options.title,
            description=options.description,
            tags=tags,
            categoryId=options.category
        ),
        status=dict(
            privacyStatus=options.privacyStatus
        )
    )
    insert_request = youtube.videos().insert(
        part=",".join(body.keys()),body=body,media_body=MediaFileUpload(options.file, chunksize=-1, resumable=True)
    )
    return resumable_upload(self,insert_request)

def resumable_upload(self,insert_request):
    RETRIABLE_EXCEPTIONS = (httplib2.HttpLib2Error, IOError, httplib.NotConnected,
    httplib.IncompleteRead, httplib.ImproperConnectionState,
    httplib.CannotSendRequest, httplib.CannotSendHeader,
    httplib.ResponseNotReady, httplib.BadStatusLine)
  
    RETRIABLE_STATUS_CODES = [500, 502, 503, 504]
    response = None
    error = None
    retry = 0
    id=False
    while response is None:
        try:
            print "Uploading file..."
            status, response = insert_request.next_chunk()
            if response is not None:
                if 'id' in response:                    
                    print "Video id '%s' was successfully uploaded." % response['id']
                    id=response['id']
                else:
                    critical("youtube:resumableUpload:unexpectedError","The upload failed with an unexpected response: %s" % response)
                    return False
        except HttpError, e:
            if e.resp.status in RETRIABLE_STATUS_CODES:
                error = "A retriable HTTP error %d occurred:\n%s" % (e.resp.status,e.content)
            else:
                raise
        except RETRIABLE_EXCEPTIONS, e:
            error = "A retriable error occurred: %s" % e
            
        if error is not None:
          warning("youtube:resumableUpload:error",error)
          retry += 1
          if retry > MAX_RETRIES:
            critical("youtube:resumableUpload:maxRetries","No longer attempting to retry.")
            return False
        
          max_sleep = 2 ** retry
          sleep_seconds = random.random() * max_sleep
          print "Sleeping %f seconds and then retrying..." % sleep_seconds
          time.sleep(sleep_seconds)
    return id
  
def run_touyube_flow(self,flow, storage, flags, httpd,oauth_callback):
    logging.getLogger().setLevel(getattr(logging, flags.logging_level))
    flow.redirect_uri = oauth_callback
    authorize_url = flow.step1_get_authorize_url()

    import webbrowser
    if (webbrowser.open(authorize_url, new=1, autoraise=True)):
      timeout=self.loginTimeout
      self.info("youtube:run_touyube_flow:webbrowserOpen","Please use the raising web browser to sign in your Google account (Timeout: {:d} secs)".format(timeout))
    else:
      self.critical("youtube:run_touyube_flow:webbrowserFailed","Unable to open Web Browser")
      return False

    code = None

    httpd.handle_request()
    if 'error' in httpd.query_params:
	self.critical("youtube:run_touyube_flow:rejected","Authentication request was rejected.")
	return False
    if 'code' in httpd.query_params:
	code = httpd.query_params['code']
    else:
	self.critical("youtube:run_touyube_flow:codeNotFound","Unable to find authentication code")
	return False
    try:
        credential = flow.step2_exchange(code)
    except client.FlowExchangeError as e:
        self.critical("youtube:run_touyube_flow:failed",'Authentication has failed: {0}'.format(e))
        return False

    storage.put(credential)
    credential.set_store(storage)
    self.info("youtube:run_touyube_flow:success",'Authentication successful.')

    return credential
  
def closeServer(httpd):  
    httpd.socket.close()
    httpd.shutdown

 

  • Quote
    • Like 1

    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.

    Guest
    Reply to this topic...
    Go to topic listing

    Important Information

    We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.