Developers Blog

Blog > News of the week - 16

on Apr 21, 2013

This week was devoted to some improvements on Tryphon Console and our infrastructure.

Console Stream improvements

Streams hosted on stream.tryphon.eu can be monitored via the Tryphon Console.

Stream live usage

The Tryphon Console has been improved. You can now follow in live the listener count for each stream :

Stream details in Tryphon Console

CSV export

Stream CSV export

You can export the stream usages in a simple CSV format to make our own statistics view.

The CSV file contains simply a time and the listener count at this time. The file contains the usages each five minute.

Javascript API

A beta javascript API is available to retrieve stream status (up/down and listeners count).

This API is available (for the moment) on http://console.tryphon.eu/streams/:id/status.js.

You just have to specify the javascript function to be called with the stream status (callback). You can specify a refresh parameter to recall the function periodically.

Here is a small example :

   <script type="text/javascript">
     function stream_status(status) {
       $('span.listeners').html(status.listeners);
     }
   </script>
   <script type="text/javascript" 
           src="http://console.tryphon.eu/streams/:id/status.js?callback=stream_status&refresh=60" 
           defer="true"></script>

This API is used on some Bonnes-Ondes sites (like L’Improbable Podcast or La Radio des Foyers) to display their stream player only when they are live.

Monitoring improvement

Our server infrastructure uses logcheck to analyse hourly our centralized log. Rules explain to logcheck what are “normal” log messages.

Our customized rules database (about 500 rules) has to be updated time to time.

Here are some rules for Rivendell log messages :

^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ caed: alsaStopTimerData\(0\)$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ rdairplay: finished event: Line: [0-9] Cart: [0-9]+ Cut: [0-9] Card: [0-9] Stream: [0-9] Port: [0-9]$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ caed: UnloadPlayback - Card: [0-9] Stream: [0-9] Handle: [0-9]+$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ caed: LoadPlayback Card: [0-9] Stream: [0-9] Name: /var/snd/[0-9_]+\.wav Handle: [0-9]+$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ caed: PlaybackPosition - Card: [0-9] Stream: [0-9] Pos: [0-9]+ Handle: [0-9]+
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ caed: Play - Card: [0-9] Stream: [0-9] Handle: [0-9]+ Length: [0-9]+ Speed: [0-9]+ Pitch: [0-9]+
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ rdairplay: started audio cart: Line: [0-9]+ Cart: [0-9]+ Cut: [0-9]+ Pos: [0-9] Card: [0-9] Stream: [0-9] Port: [0-9]$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ caed: FadeLevel: [0-9]+$
 
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ rdairplay: chained to log: Line: [0-9]+ Log: [[:alnum:]_]+$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ ripcd: received rml: 'LL 1 [[:alnum:]_]+ -2!' from 127\.0\.0\.2$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ rdairplay: loaded log '[[:alnum:]_]+' in Main Log$
 
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ ripcd: ran local maintenance routines$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ ripcd: ran system-wide maintenance routines$

We created a small helper tool logheck-rule.sh to make easier rule tests :

$ ./script/logcheck-rules test
Apr 22 06:26:24 radio rails/audiobank[5072]: Processing CastsController#play to smi (for X.X.X.X at 2013-04-22 06:26:24) [GET]
Apr 22 06:38:19 radio backup[10914]: Syncer::Cloud::S3 started the syncing process:
Apr 22 06:38:19 radio backup[10914]:   Concurrency: false Level: 2
Apr 22 06:38:19 radio backup[10914]:   Generating checksums for '/srv/audiobank/originals'
Apr 22 06:38:19 radio backup[10914]: 'audiobank/originals/1'
...

$ grep "rails/audiobank" syslog | ./script/logcheck-rules filter
Apr 22 06:26:24 radio rails/audiobank[5072]: Processing CastsController#play to smi (for X.X.X.X at 2013-04-22 06:26:24) [GET]
...

You can use this tool to view syslog with your logcheck rules :

$ ssh monitor.tryphon.priv tail -F /var/log/syslog | ./script/logcheck-rule filter | lwatch -i-