Building our network, improving products to solve our own information problems, maybe this will help you too.

At gobisoft, we’re constantly on the road and several of our employees work away from the office, so keeping a semi-centralised (the ’semi’ bit I’ll talk about in another post) location for our data is important.

We started by having folders on our own desktops and a ‘project’ directory on one of our webservers for common documents which we would access over SFTP. But this means we needed to use SCP/SFTP clients like WinSCP and Transmit.

Doing this became a pain in the neck because when you’re in Excel or Quickbooks and you’ve created a document, you have to save it to your computer, copy it to the server, then open it on the server to continue editing. Using simple Windows / SMB networking means that your first save can be onto the server.

So we went about researching how to use Windows networking over the internet. And we started with Virtual Private Networking (VPN) and a freeware product called ‘Hamachi‘. But we didn’t like the way it made you use the 5.x.x.x ip address range on our network, and we didn’t like the speed performance. It’s a great product but we didn’t want to run SMB over TCP/IP over PPTP over TCP/IP. With running over SSH, you’re cutting out a step, SMB over SSH over TCP/IP, thus it ‘felt’ faster (try it your self and see what you think).

Once we rules VPN out, we looked at SSH. We had done SSH tunneling before but never at the windows networking level, mostly just tunneling HTTP.

Our findings on how you can connect to a Samba server over SSH are applicable to Mac OSX:
Connecting to the gobisoft File Server (Mac)

and Windows XP:
Connecting to the gobisoft File Server (Windows XP SP2)

We’re pretty sure that with a bit of tinkering you could do the same in Windows 2000 and Linux.

Far from perfect

If you read the documents, you’ll immediately notice the difficiency in both articles and that is that you need to turn off Windows File Sharing on both your Windows XP box and your Mac. For us at the moment, this isn’t too much of a problem since we use our own personal computers which weren’t bought by our company but in the future, when we expand, this will be a issue since we’ll want to access Windows Shares on all the computers that are connected to the LAN as well as our WAN server.

Enter Flow-sync

We’re currently working on a solution using virtual machine based technology which enables employees to connect to each other over their LAN and to our server at the same time.

If you’re away from the internet or you’re over a really slow DSL connection, programs like Word and Excel freeze when they’re saving, affecting your computer stability, so our second benefit of using virtual machine technology is that your computer can save to the virtual machine, which runs Flow-sync (our in-development product). Flow-sync will then update the saved changes to the server when it’d needed. For all files including Photoshop files, Flow-sync will only transfer what’s changed, making saves a lot quicker.

The next part of Flow-sync

Also important is that the solution needs to take account of people editing the same file at the same time. We achieve this through simple file locking techniques, however there is an option for certain users to unlock the file, and do a differential comparison between files to see what’s really changed.

The final part of Flow-sync

Flow-sync is part of our wider distributed working strategy, and only a small part of Flow, our document management system. Flow-sync introduces the ability to synchronise files across your (and our) Flow servers which gives us our backup redundancy. Naturally Flow (already does) and Flow-sync will export to tar archives and compressed file dumps so you can burn your backups to DVD.

This entry was posted on Monday, July 30th, 2007 at 10:20 pm and is filed under Technical. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.