Miguel Caamano Posted April 15, 2020 Share Posted April 15, 2020 Hello, After tinkering a bit with workflows, there is a feature i believe it is necessary: "on the fly" option for tasks that involve any kind of transcoding. At the moment, when trying to convert to a file a new location from "X" to "Y" format we are forced to choose between the following 2 ways: Case 01) - read from storage 01 -> convert on storage 01 -> move to storage 02 Case 02) - read from storage 01 -> copy from storage 01 to 02 -> convert on storage 02 -> delete copy on storage 02 It might seem pretty simple, but both scenarios create their own issues that could very easily solved with a "on the fly" option: Case 01: Seems pretty straightforward, but there is a problem: The "original file to format Y " conversion creates traffic both ways (reading and writing) to the storage. And then a third time to move the conversion out of its original place. This is x3 amount of traffic on the network. In a 10gbE that might not be a problem, but on a WAN or even a gigaBit ethernet, that can be a show stopper. Also, moving is a kind of "deleting" operation, that might not be "wise" on certain types of environments. Case 02: I believe this one will be the most usual, but still shows issues: The original file gets copied to the second storage location, and then read again to be transcoded. Later, the copy of the original file gets deleted. This creates, again, a x3 the amount of needed traffic on the network and temporarily x2 on the storage space. In both scenarios the process of conversion is going to work, but the stress the nertwork suffers is worthless. In a few hundred megabytes file this is alright, no harm. But going to large sizes and large quantities of files, this becomes slow-to-almost-unusable. Also, we may think that network & storage resources are always scarce. The solution, in my opinion, would be to create a "on the fly" option, that uses the system internal memory, to handle the conversion without relying on the storage as such. Broadly speaking, this would force the workflows to: read the original file -> store it, or a portion of it, on its memory for transconding -> transcode -> copy/render it to the destination location. I believe that this should be an option for all the nodes involved in converting/transcoding/transforming any material. And that should be exposed on the face of the node, when the option is activated Thanks a lot. Miguel. Link to comment Share on other sites More sharing options...
Cristobal Bolaños Posted April 16, 2020 Share Posted April 16, 2020 Hi Miguel! Thanks a lot for this feedback. I have proceed with this as with the other features. However, I have some doubts regarding this feature. I mean, when transcoding from storage 01 to storage 02, you are not necessarily forced to make a copy first. You can directly send the transcode to be placed on a different storage. Have you tried that? Cheers, Cristóbal Link to comment Share on other sites More sharing options...
Miguel Caamano Posted April 16, 2020 Author Share Posted April 16, 2020 Hi Cristobal. My Bad completely, I got two things mixed up in this post. Please disregard it completely. Will elaborate in a new one. Apologies. Miguel. Link to comment Share on other sites More sharing options...
Recommended Posts
Please sign in to comment
You will be able to leave a comment after signing in
Sign In Now