rTorrent 0.8.4 and ruby-controller 0.2

Yesterday i switched to rtorrent 0.8.4 (debian build from experimental).

Rtorrent after some little adjustment in the configuration file has started working. I just had some problems with the torrent which i hadn’t completed.

Except these two little problems it just works, along with the ruby controller too (without any modification).
In the meanwhile I have added some little fixes to the controller and a new feature. 
Now the available upload bandwidth is decreased by a fraction of the actual rtorrent download speed.
This is done to prevent even further to rtorrent to make itself slower flooding the upload bandwidth with ack packets.

Ruby rtorrent controller

Finnaly i had time to work on the rtorrent controller.
it works, that’s all i can say everyone will have to test it and fix the various values to setup the network for each own best performances.

There are many costants that may be configured to solve problems you may encounter with the configuration. Everything should be clear in the comments but i’ll try to add  more comments anyway when i have time.

With the configuration in the distribution (made for myself for my 2M adsl) i can have my sister doing video chats and playing online without any lag problem and when everything is finihed rtorrent autmatically will raise his own bandwidth limits.

There is currently a drawback. Rtorrent versions prior of 0.8.1 have a memory leak regarding xmlrpc-rtorrent communications. This means that rtorrent is going to eat all your ram. I am using the program monit to monitor my own servers and it’s  just trivial with it to keep rtorrent in a sane memory usare buy restarting it everytime it goes too far. Otherwise even a cronjob will do the trick restarting rtorrent every 1 or 2 days.

You can find the link to the project in the code page or just here
git://github.com/mellon85/ruby-rtorrent.git

mupnp updated

mupnp has been updated to version 0.1.1.

The point is that i was experimenting problems with firehol firewall wich had UPnP ports enabled but I still wasn’t retriving any answer.

To fix this i had to hack a little on miniupnp library and finished searching when i found the TX_FROM_UPNP_PORT constant wich wasn’t set by default due to compatibility problems with Windows XP. I just made a slight change and converted it in a runtime check instead of a compile time one. This changed the C api of the library for the upnpDiscover function, and added a value to the initialization of the high level interface written to interface with the library which is automatically set to send data from the upnp port.

I have sent the patch to the mainstream developer, but it would require api changes and more modification to make it work correctly on the full fledged library.

With this last fix the rtorrent ruby controller is almost ready, i just have to comment the code and write a post about it

The gem is already available for upgrade on the rubyforge webstite or gem utility.

UPnP Gem!

I have just discovered the power of the mkmf package in Ruby.
It simplify the way to build the upnp module a lot!
I just had to move the files to be compiled in a folder (called miniupnpc) and then just call this

require 'mkmf'
create_makefile("MiniUPNP","miniupnpc")

And i’ll have a makefile ready to compile the module, no matter which is you platform! With theis new knowledge and the immense usefulness of the ferret’s Rakefile i’have built and published a ruby gem that will install the UPnP module and the wrapper.

The name is simply mupnp. Installable with the gem utility. For windows the library must be precompiled, so it may not be immidiatly upated or released when needed.

There have been some API changes, to make it more ruby style, anyway it is nothing big, just check at the documentation of the methods and check for differences.
[Only addPortMapping and the initialize method had a change in the order of the arguments]

To install it just do:

gem install mupnp

And it will be downloaded and compiled on your mac/linux box. For windows version i’ll have to find someone to trust that will build it for me.

UPnP – Ruby Wrapper

[EDIT:This has been released as gem. check the code page]

[Continues: UPnP Integration]

After some testing i wrote an UPnP class to wrap the UPNP library for a more friendly and usable interface.
The wrapper class can be found on the same code repository.

I have added a nice feature too, adding the discovery of UPnP devices as a thread on object creation, to let it probe the network without bothering the class user. But this lead me into troubles… the thread that was going to probe the network made ruby wait until it is completed, even if it has to wait on a socket… digging for the threading implementation of ruby i finally got where the problem was.
Ruby uses green thread, cooperative user threads… that was the problem.. every thread has to tell which other thread is the next. But what happens if it remains blocked on a system call? it simply makes the whole ruby machine to wait … fortunately ruby 2.0 should solve this problem, requiring that the threads must be operative system threads.

 

The Initializer allows you to choose how much time should the implementation wait for an UPnP answer from the network, and if the network probing thread must be launched. Remember that the object must be initialized with a discoverIGD call, except if you let the thread start.
Initialization functions

• initialize(max_wait=1000,autodiscover=true)
• discoverIGD()

IP functions

• getLanIP()
• getRouterIP()
• getExternalIP()

Router info

• getStatusInfo()
• getConnectionTypeInfo()

Traffic info

• getTotalBytesSent()
• getTotalBytesReceived()
• getTotalPacketsSent()
• getTotalPacketsReceived()
• getMaxLinkBitrates()

Mappings

• getPortMappings()
• getPortMapping(nport,proto)
• addPortMapping(nport,lport,client,desc,proto)
• deletePortMapping(nport,proto)

The port numbers may be passed as integer or string, and they are always returned as integer (lib miniupnp works only with strings). All the errors are managed through exceptions.
In fact there is a nice thing about ruby threading, the exceptions that occour in a thread are received by the thread that calls join on the thread. Due to this, the first method you’ll call (except for your ip and router ip functions), if the autosearch feature is enabled, will return the exception, if any, from the detect thread.
Uh, i was just about to forget this. nport is the net port, and lport is the lan port.
This is a simple example that will print each mapping in the router. The call to getPortMappings will wait until the discover thread has finished searching a router.

require 'upnp'
u = UPNP.new()
u.getPortMappings.each {|m| puts m.to_s}

 

 [EDIT:This has been released as gem. check the code page]

UPnP – Ruby Integration

[EDIT:This has been released as gem. check the code page]

At the moment i’m trying to create a ruby module to integrate some upnp functionality in the language.

I would like to do so to enhance the experience with rtorrent, a really nerdish torrent client. Ok, it uses curses, as interface, but it is just great, never crashed in 6 months and had a tiny memory footprint.

The idea is to connect to the local router and gather information about the port mappings and the data currently flowing in the local network and trough estimation change the upload rate.

Useful to use the bandwidth when it is unused, as long as i pay it i want to use it!  Here is how I have built such a module on Mac OS X 10.5.3 with the apple’s version of ruby (1.8). To avoid to write the code i’ll use the miniupnp (1.0) library and the utility swig (1.3.35)

The first is a upnp library which interacts with the IGDs (Internet Gateway Device) and the latter is an wrapper builder that will write all the glue code (CRuby) in my place.

Get miniupnp to compile

The first thing is to get the library to compile on my mac. The distribution makefile has some minor flaws, I have modified it to build the dynamic library with this patch makefile-osx.patch

--- a/Makefile
+++ b/Makefile
 HEADERS = miniupnpc.h miniwget.h upnpcommands.h igd_desc_parse.h \
 upnpreplyparse.h upnperrors.h
 LIBRARY = libminiupnpc.a
-SHAREDLIBRARY = libminiupnpc.so
+SHAREDLIBRARY = libminiupnpc.dylib
 SONAME = $(SHAREDLIBRARY).$(APIVERSION)
 EXECUTABLES = upnpc-static upnpc-shared \
 testminixml minixmlvalid testupnpreplyparse
@@ -83,7 +83,7 @@ $(LIBRARY):	$(LIBOBJS)
 $(AR) crs $@ $?

 $(SHAREDLIBRARY):	$(LIBOBJS)
-	$(CC) -shared -Wl,-soname,$(SONAME) -o $@ $^
+	$(CC) -dynamiclib -o $@ $^

 upnpc-static:	upnpc.o $(LIBRARY)
 $(CC) -o $@ $^

As the build completes everything is almost done. Another patch has to be applied to avoid a swig warning for a memory leak (miniupnp.g-memleak.patch).

--- a/miniupnpc.h
+++ b/miniupnpc.h
@@ -16,7 +16,7 @@ extern "C" {
 #endif

 /* Structures definitions : */
-struct UPNParg { const char * elt; const char * val; };
+struct UPNParg { char * elt; char * val; };

 int simpleUPnPcommand(int, const char *, const char *,
 const char *, struct UPNParg *,

Get Glue Code

Then we have to write a swig interface file, describing what should be put in the ruby module writing this in the file upnp.i

%module miniupnp
%{
#include "miniupnpc.h"
%}

%include "cpointer.i"
%pointer_functions(unsigned int, uintp);

%import declspec.h
%include miniupnpc.h
%include upnpcommands.h
%include igd_desc_parse.h

Make the Module

This will create a module that will contain all the functions from miniupnpc.h, upnpcommands.h and igd_desc_parse.h and a struct i have added to ease the use of the library. Swig is then called

swig -ruby upnp.i
gcc -fPIC upnp_wrap.c -c -o upnp_wrap.o -I/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/universal-darwin9.0/

Generating the file upnp_wrap.c glue code which must be compiled with the ruby includes. From this point there is a decision to take, if the miniupnp library is to be linked to the module as an external shared library or be included in it as static code.

To link it dynamically

gcc -fPIC -bundle -undefined dynamic_lookup upnp_wrap.o -o miniupnp.bundle -lruby -lminiupnpc -L<miniupnpc install path>

And to link it statically

gcc -fPIC -bundle -undefined dynamic_lookup upnp_wrap.o -o miniupnp.bundle -lruby miniwget.o minixml.o igd_desc_parse.o minisoap.o miniupnpc.o upnpreplyparse.o upnpcommands.o minissdpc.o upnperrors.o

Test it

Now is time to use the newly created module. This can be done in 2 ways too. Installing it in the ruby library path or configuring the library path.


# system install
cp miniupnp.bundle /Library/Ruby/Site/1.8/
# setup path, bundle_path is where the
# bundle is (for instance $PWD)
export RUBYLIB=bundle_path


Here there is a little script to test that everything is working

# The feature name begins with a lowercase letter...
require 'miniupnp'

# max time to wait for the upnp devices
MAX_WAIT_TIME = 1000

# discover upnp
list = Miniupnp.upnpDiscover(MAX_WAIT_TIME,nil,nil)
if list == nil then
    puts "No UPNP device found"
    exit 1
end
puts "URL : #{list.descURL}"

# get valid internet gateway device
urls  = Miniupnp::UPNPUrls.new
datas = Miniupnp::IGDdatas.new
lan = ""*16
r = Miniupnp.UPNP_GetValidIGD(list,urls,datas,lan,16)
case r
when 0
    puts "No IGD found"
    exit 2
when 1
    puts "A valid connected IGD has been found"
when 2
    puts "A valid IGD has been found but it is reported as not connected"
when 3
    puts "An UPnP device has been found but was not recognized as an IGD"
    exit 3
end

puts "Client LAN IP: #{lan}"
ext=""*16
r = Miniupnp.UPNP_GetExternalIPAddress(urls.controlURL,datas.servicetype,ext)
if r == 0 then
    puts "External IP: #{ext}"
else
    puts "Error while retriving the external ip address"
end

# print urls
puts "*** urls"
puts "controlURL: #{urls.controlURL}"
puts "ipcondescURL: #{urls.ipcondescURL}"
puts "controlURL_CIF: #{urls.controlURL_CIF}"

# print datas
puts "*** datas"
puts "cureltname: #{datas.cureltname}"
puts "urlbase: #{datas.urlbase}"
puts "level: #{datas.level}"
puts "state: #{datas.state}"
puts "controlurl_CIF: #{datas.controlurl_CIF}"
puts "eventsuburl_CIF: #{datas.eventsuburl_CIF}"
puts "scpdurl_CIF: #{datas.scpdurl_CIF}"
puts "servicetype_CIF: #{datas.servicetype_CIF}"
puts "devicetype_CIF: #{datas.devicetype_CIF}"
puts "controlurl: #{datas.controlurl}"
puts "eventsuburl: #{datas.eventsuburl}"
puts "scpdurl: #{datas.scpdurl}"
puts "servicetype: #{datas.servicetype}"
puts "devicetype: #{datas.devicetype}"

puts "Get maximum bandwith"
up=Miniupnp.new_uintp()
down=Miniupnp.new_uintp()
Miniupnp.UPNP_GetLinkLayerMaxBitRates(urls.controlURL_CIF,datas.servicetype_CIF,down,up)
puts "up: #{Miniupnp.uintp_value(up)} down: #{Miniupnp.uintp_value(down)}"
Miniupnp.delete_uintp(up)
Miniupnp.delete_uintp(down)

# release memory
Miniupnp.freeUPNPDevlist(list)
Miniupnp.FreeUPNPUrls(urls)

I can’t get wordpress to display correctly the content of lan and ext variables, it should be “” repeated 16 times. The result should be something like this

URL : http://192.168.0.1:49152/gateway.xml
A valid connected IGD has been found
Client LAN IP: 192.168.0.100
External IP: ***.***.***.243
*** urls
controlURL: http://192.168.0.1:49152/upnp/control/WANIPConnection
ipcondescURL: http://192.168.0.1:49152/ipcfg.xml
controlURL_CIF: http://192.168.0.1:49152/upnp/control/
WANCommonInterfaceConfig
*** datas
cureltname: presentationURL
urlbase: http://192.168.0.1:49152
level: 0
state: 3
controlurl_CIF: /upnp/control/WANCommonInterfaceConfig
eventsuburl_CIF: /upnp/event/WANCommonInterfaceConfig
scpdurl_CIF: /cmnicfg.xml
servicetype_CIF: urn:schemas-upnp-org:service:WANCommonInterfaceConfig:1
devicetype_CIF: urn:schemas-upnp-org:device:WANDevice:1wayDevice:1
controlurl: /upnp/control/WANIPConnection
eventsuburl: /upnp/event/WANIPConnection
scpdurl: /ipcfg.xml
servicetype: urn:schemas-upnp-org:service:WANIPConnection:1
devicetype: urn:schemas-upnp-org:device:WANConnectionDevice:1

Maximum bandwith
up: 352000
down: 2464000

This is just a start, I’ll make more testing and i’ll make sure it works. There may be some more things to add to upnp.i

Full code can be found in the code page
Edit 15-06-08: Module name changed to miniupnp from upnp

[Continued by: UPnP Wrapper]
[EDIT:This has been released as gem. check the code page]