DTube Uploader, born to be alive!

Before I start talking about the uploader, I have to talk to you about how I joined the DTube team.
It goes back to 6 years ago! (Yes I know DTube to less than 1 year ^^).
It started with the meeting of @Heimindanger as an employee both in a company where have developed an eCommerce website.
We worked together a good year before he flies his own wings!
We kept a good distant relationship.
One day he contacted me and asked me if I wanted to participate in the project, he explains the project, I did not know Steem, IPFS, Meteor, crypto-currencies... I thought about it and I said yes!

The different versions

I started contributing to DTube on version 0.3 4 months ago with the possibility of translating DTube into several languages https://steemit.com/dtube/@heimindanger/dtube-0-3-this-update-contains-so-many-new-features-i-could-not-find-a-good-title-for-it.

Then, I hooked up with the creation of any part of the uploader for version 0.5 only 2 months ago https://steemit.com/dtube/@heimindanger/d-tube-0-5-big-optimizations.

Finally the possibility to have a preview of the video on the scrollbar, as well as the ability to send the Snap to resize and embed the new logo DTube, there is 1 month in v0.6 https://steemit.com/dtube/@heimindanger/d-tube-0-6-pushing-it-to-the-limit.

My first article!

If you followed, I'm quite new to Steem and it's also my first article on the internet, thanks in advance for your indulgence ^^

The subject of this article is about the uploader! I decided to do this article because many people are wondering about the operation, I thought it was a good reason to dive into the bath for this first article!

Why uploader?

To answer problems that users were facing and because it was not possible easily in decentralized mode!

The first need was to encode the source videos in a smaller format (480p) allowing more people to play videos with a low internet speed.

The 2nd was to have the possibility to have a progress bar when sending the video file.

We then took advantage (having on hand the source video!) to encode the video in a video and audio format allowing to be read by the largest number of browsers on the market (H264 for video and AAC for audio).
And this allowed us to take the opportunity to directly send the source video and 480p video directly into IPFS and thus lighten the customer of this long and unstable task at the time.
In version 0.5, we then added the creation of a video Sprite allowing to visualize thanks to the scroll bar of the video the preview of the moment of the video.

Let’s go!

Here is the block-by-block explanation of the operation of the uploader that allows you to send videos to DTube!

Upload video

b/ Sending the video to the server with progress bar
f/ Token recovery for encoding tracking, IPFS hash recovery, ...

The heart of the Uploader

For fast processing after the end of the upload, everything is treated as asynchronously and in parallel, there is only one token that is returned with prior requests for creation of Sprite (d), encoding (e) performed …

Encoding Daemon

Daemon encoding can handle 2 different tasks, creating Sprites files and 480p encoding of the source file.

How the Sprite is created to preview the video:

• Extraction of 100 images of the source video in 210 * 118 respecting the ratio
• Combining 100 images vertically => allows to see the first images if the image is slowly loading!
  Note: if the video is less than 100 seconds, the extraction of images is done with 1 image per second, which is more than enough.

For 480p encoding operation, the source video is simply converted to 854 * 480 for 16: 9 video.
Video conversion to H264 is also performed as well as audio conversion to AAC. Thus this 480p video can be read by many people and by several browsers, unlike the source video.

Once each file is ready (Sprite or 480p file), it is placed in the IPFS queue.
The progress of each treatment is updated every second so that the user can follow it.

IPFS Daemon

The file is added to the decentralized IPFS network.
Once the user recovers all the hashes via repeated calls, the video article can be posted on the Steem network.

Finally on the home page

The video list is extracted from Steem's blockchain, and each video contains an url link from IPFS, hosting the Snap, video, Sprite...

Global scheme !

Extra: the Snap and the DTube logo inlay

Steps:

• Simultaneous scaling and cropping for a final image of 210 * 118
• Adding in IPFS this final image
• Adding IPFS to this image with the DTube logo overlay

Optimizations

If a user had to leave the upload page after sending the video, 480p video encoding and adding to IPFS would be useless and valuable resources would be lost.
That's why if the upload page is closed, the tasks associated with this source video on the video encoding, the creation of the Sprite, the addition in IPFS ... are canceled!
This point saved a lot of resources several weeks ago.

Limitations

The 480p encoding and the creation of the Sprite is for the moment limited to videos less than 20 minutes for cost reasons. Which is still the majority of videos sent on DTube.
The encoding of the source video is currently limited to 480p for cost reasons as well.
Note: These are things that will evolve and change in the future!

The source video

The source is kept as is, without any transformation and limitation in format, size, duration ... but we must pay attention to browsers and Internet speeds of users who may not be able to view the video and not vote at once!
A video with a smaller size, and an encoding in h264 for video and AAC for audio will allow all to see your source video, think about it!

Shortly

The uploader is evolving!
Since the last version v0.6 things have accelerated, the affluence helping and the technical challenges that grow with, and it's rather motivating!
The next article will explain these changes.

Conclusion

This is my very first article that I publish on the internet and also on the blockchain Steem, be indulgent!
Thanks to @heimindanger for his tips for writing articles!
Feel free to leave your comments. I’m french, so thanks to google translate!

Links

DTube: https://d.tube
Uploader: https://github.com/dtube/ipfs-uploader