User Tools

Site Tools


howto_configure_run_peerstreamer_in_cloudy

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

howto_configure_run_peerstreamer_in_cloudy [2015/01/24 12:14] (current)
felix File build by odt2dw plugin from filePS_Clommunity1.odt
Line 1: Line 1:
 +====== PS_Clommunity1 ======
 +<​sub>​{{PS_Clommunity1.odt|Original file}}</​sub>​
 +
 +**PEERSTREAMER ON CLOUDY**
 +
 += Introduction =
 +
 +Peerstreamer is a peer-to-peer video streamer. When a peer starts a streaming and some other peers connect to it, a mesh network is created between the different peers. The source divides the video in chunks and sends these chunks to different peers. The peers exchange the chunks between them so they create a peer-to-peer streaming.
 +
 += How Peerstreamer works on Clommunity =
 +
 += Create a stream =
 +
 +In order to create a streaming, the first thing we need is a video source. It can be a video or an already existing stream in the internet (like an online TV channel). Once we have the stream, we must go to Clommunity → Peerstreamer. There we can see two buttons: one joins an already existing stream (in the peerstreamer overlay), and the other one to creates a stream. To create a video, the seconf button must be clicked.
 +
 +{{howto_configure_run_peerstreamer_in_cloudy_Image_0.png}}
 +
 +Figure1: Peerstreamer screen and Clommunity menu.
 +
 += From a video =
 +
 +If you want to stream a video file located in your local drive, thefirst thing to do is to provide a stream with this video. This is mandatory since the Peerstreamer interface from Cloudy only supports streams as an input but not files. There are many ways to do this, but a very easy one is using the well known VLC, which allows to create streams and bind it to any port (and IP) we have available.
 +
 += From an existing online stream =
 +
 +Once we have the stream we want to share in our community network using Peerstreamer,​ the only thing we need to do is to insert the stream’s URL in the field with the tag URL Source. Afterwards we must assign a port to our channel (this number will be our stream ID in the community network) and a brief description of the channel. The only thing left to do is clicking on Publish.
 +
 +{{howto_configure_run_peerstreamer_in_cloudy_Image_1.png}}
 +
 +Figure 2: Publishing a video from a stream with some example port and description
 +
 += Connect to a stream =
 +
 +There are two ways to connect to a stream: knowing its IP and port, or using the avahi technology. By default, We are using the avahi technology since it simplifies the process and does not force you to know any technical data about the source. However, if the video we want to watch is streamed in another microcloud, we will need some info about some source (this last option is useful if we want other people in our microcloud to watch this channel by getting its info from avahi). The two options are explained below.
 +
 += Getting the stream from Avahi =
 +
 +If we want to join a video in our microcloud, we just need to go to Searchi → “Look for services in µCloud”. This process might take a few seconds to complete, but once it’s done, a list of services will be shown. There are different tags depending on the type of service existing in our microcloud. If there is any peerstreamer instance, a “Peerstreamer” tag will appear. Once we have clicked the tag, a list of video streams will be provided. Every element (i.e:​stream) of this list has a “Join now” button besides it. As the button says, We just need to click there to configure some last parameters to finaly join the stream.
 +
 +{{howto_configure_run_peerstreamer_in_cloudy_Image_2.png}}
 +
 +Figure 3: Avahi service list
 +
 += Getting the stream from a different microcloud source =
 +
 +If we want to join a video that is only (avahi)published in another microcloud, we need the IP of some machine in that microcloud and the stream port. If we have this, we can click on the button “Connect to a Peer” in Clommunity → Peerstreamer. The following window asks us for some parameters (the first two will be already filled if we come from avahi). The first one asks for the source IP, the second asks for the Port.
 +
 +{{howto_configure_run_peerstreamer_in_cloudy_Image_3.png}}
 +
 +Figure 4: How to access the page to connect to external source
 +
 +The third one must be RTSP if the video is going to be watched by more than one people connected to the node (router), otherwise, you can set it to UDP. Finally, we just need to provide an internal port to restream the video (whichever we want as it is always greater than 1023).
 +
 +{{howto_configure_run_peerstreamer_in_cloudy_Image_4.png}}
 +
 +Figure 5: Filling data from an external stream source
 +
 +If you click to connect, you’ll see the video stream in the vlc web plug (if installed and if previously you set UDP and not RTSP), and a link to watch the video from any video program that allows external stream (like VLC).
 +
 +{{howto_configure_run_peerstreamer_in_cloudy_Image_5.png}}
 +
 +Figure 6: Peerstreaming running with the web plugin
 +
 += Stopping a (re)stream =
 +
 +Stopping a stream or a re-streaming is very simple from the Cloudy web interface. You just need to go to Clommunity → Peerstreamer. The Peerstreamer instances that you initialized will be shown in a list there. You will see two options: “View” and “Stop”; so if we want to stop the stream, we just click on “Stop”.
 +
 +{{howto_configure_run_peerstreamer_in_cloudy_Image_6.png}}
 +
 +Figure 7: Stopping a video stream re-publish
 +
 += Changes and implementation =
 +
 += pscontroller script =
 +
 += Installation process =
 +
 +First thing to be done was installing the requiered software for Peerstreamer to work, this is, peersteramer itself and vlc-nox (no Xserver VLC version). The function doInstall was created with its corresponding check functions. Note that peerstreamer is compiled for some different architectures,​ such as i386 and amd64 (including arm).
 +
 += Kill zombie processes =
 +
 +While making some tests, we realized that some times peerstreamer crashes for some unexpected error, leaving useless VLC processes running. We solved this using the pspeers.conf file that Cloudy creates when you join a stream. This file contains information about the source of the stream, the port which is being used, PS PID, VLC PID, and some other useful parameters. ​
 +
 +The function “doInfo” was modified so everytime it is called, it checks all the instances in that file and kill the inactive processes. Note that in the case of the Peers, there is no VLC instances. In this case, VLC PID is set to 0 (the case is considered before killing to avoid killing all group processes).
 +
 += Avoid channel redundance =
 +
 +Besides of the overhead it suposes, Cloudy may get frozen if multiple peerstreamer instances listen to the same channel (this is, same port) in the same node. To avoid this situation, whenever a node connects to a stream, it checks if there is already an instance connected to this channel. So the port can be seen as some sort of primary key in the pspeers.conf file.
 +
 += Delete pspeers.conf file when empty =
 +
 +We added the functionality to delete the file when the last instance is deleted (i.e: last peerstreamer instance is stopped) just to keep things clean.
 +
 += VLC web plugin =
 +
 +When making some tests, we realized that the VLC web plugin was not working well. We didn’t see any video altought the peerstreamer was working well. We could check this by getting the internal stream link and putting it to the vlc. Finally, we found out that the problem was because of the internet browser (we were using chrome) since the plugin worked really good on Iceweasel.
 +
 += Make Avahi get information from pspeers.conf file =
 +
 +In order to simplify the avahi process, we thought that it would be good if it could get the info from the already existing file. To do this, it was necessary to add the channel description to the file, so we medified the functions which writes to pspeers.conf so they included the channel description.
 +
 +Next step was to modify avahi so it could work with this new idea. First of all, we needed a way to provide it with the info in a “readable” format. The flag “avahi” was created so that pscontroller’s info option returns the channel description,​ a type (peerstreamer) and the channel.
 +
 +__peerstreamer.service and ps_shell__
 +
 +The only thing left was to implement a new avahi service file called peerstreamer.service,​ which had the same facilities than the other services. The difference with avahi is that it has multiple instances instead of one, so the file is a little different than the other *.service files.
 +
 +In addition, the peerstreamer.service file needed to interact with the pscontroller script. Since the requiered modifications would have been hard and extense, we decided to create some kind of shell around the pscontroller which acts as an intermediate between the controller and the service to ease the process.
 +
 +As we said before, since peerstreamer is a multi-instance service, some avahi functions make use of the ps shell. This functions in particular are “start” and “check”. Strictly, only “check” interacts with the shell, but as “start” is also using “check” we are including both of them. 
 +
 +When ps_shell is called to do a check, it stops de zombie processes (VLC instances with its Peerstreamer instances dead) if there is some and afterwards unpublish those ones. This is done by calling the “info” function in the pscontroller script with a “--no-stop” flag so it does not unpublish the services itself (the scripts needs to do it when a Peerstreamer instance crashes). Before doing this, it creates a copy of the file pspeers, so it can compare it with the original after calling pscontroller’s “info”. Then it just makes a diff and unpublish the different lines.
 +
 +When peerstreamer.service is called to do a “start”,​ it has to make a check first so it does not publish dead instances.
 +
 +The script pscontroller uses the shell too in order to publish a peerstreamer instance when it is created, and stop publishing services that are dead or stopped. This is why we need the “--no-stop” flag when calling info from the shell.
 +
 += Peerstreamer timeout =
 +
 +In Cloudy, there is a service that allow users to create microclouds;​ it is called “getinconf”. When a user is in a microcloud, some Cloudy services work only in this kind of VPN which results from using getinconf. When using this service, we have to keep in mind that the nodes IP rank is the same as the rest of nodes in the community, even that some services work just within the microcloud. This case has an effect on the relation between avahi and peerstreamer. Until now, when a peerstreamer instance crashed, we stopped the avahi publication and killed the remaining processes in the node. When the nodes connected to that peerstreamer instance notice the service stop through avahi, they automatically stop their services. However this information is only transmitted within the microcloud, since avahi is operating in it using getinconf, but external nodes won’t notice this fact.
 +
 +Remember that it is possible to join a channel by knowing its node IP and the channel port. This allow foreign nodes to join a channel anounced within a microcloud (let’s call this microcloud A). When this happens the foreign node publish this service in its microcloud (B) so other nodes can connect to it making him act like a source. The problem comes when the Peerstreamer instance in A crashes for some reason. The nodes in A would notice this crash because of avahi and would stop their instances, but nodes in B won’t notice that. This is a big problem, since it causes avahi to have a lot of useless publications,​ and may make other users to join an already dead channel. To solve this, we proposed some solutions:
 +
 +  - **UDP trap:** This was a short program in C which function was to get peerstreamer packages from a port and dump them to an internal port where peerstreamer would be listening. This was a good idea since it would allow us to monitor peerstreamer traffic in a very reliable way among other facilities, but we discarted it (for now) since it was harder than other solutions.
 +  - **Modify Peerstreamer:​** The idea was to modify the Peerstreamer source code by adding a default timer (~5 minutes) so if the program does not receive any chunk within this time, it stops. Peerstreamer has several package type, such as messages, chunks… we just counted as valid packages those that has a video chunk. This idea was not suggested at the beginning since it means that the compiler script needs to be modified too, but as it is quite simpler, we choosed this one.
 +
 +
 +Now, if a Peerstreamer instance dies, other instances external to the original microcloud will wait for 5 minutes to receive a chunk. If in this time no chunk has been transmitted,​ the instances will die and so will the avahi publication.
 +
 +=  =
 +
 += Peerstreamer compiler =
 +
 +In the Peerstreamer compiler script, the source code is taken from a git repository from the software developers. This means that to include our changes in a dynamic way we had to do something different than create a new repository. What we did is given the modified source code (we just modified the loop.c file) we made a git diff and stored the result into a file. In the compiler directory, we added a folder called patch, where patched files must be located following the same structure as in Peerstreamer source code. This is, if a file (original) is located at $PSDIR/​Streamers/​file.c,​ the diff file must be at $COMPILER/​patch/​Streamers/​file.c (filename can be different).
 +
 +If this is done well, the doPatch function in buildps script will do the rest of the job. The doPatch function is a recursive function: it explores the patch directory finding diff files and patching it to the source code files. When this is done, it proceeds with the compilation.
 +
 +With this, we achieve a modified peerstreamer executable with the modifications we’ve implemented. Note that by doing it this way, new changes can easily be implemented on the source code, dince we just need a git diff file and place it at the correct place to make the compiler works.
 +
 +
  
howto_configure_run_peerstreamer_in_cloudy.txt · Last modified: 2015/01/24 12:14 by felix