Preventing kicks because of inactivity?

Feb 21, 2013 at 6:25 PM
Hey,

first of all im really thankful for this client and the work the programmers did!

I tested the client with FileZilla Server and recognized that I got kicked after being inactive for a few minutes. A short while ago I used FtpRush and noticed an option that would keep the connection alive with sending commands like NOOP or CWD every X seconds.

Sadly the IsConnected option returnes the wrong value when the server kicks the user, seeming the socket "m_socket" in "FtpSocketStream" doesn´t notice anything, too.

Instead of using the Execute("NOOP"); command in a try-catch-block which returns a boolean when the last action was a few minutes ago, I thought asking here would be more useful, because I might have overlooked something.

Can you tell me whether there is a better way of keeping the connection alive or recognizing a disconnect/kick from the server or do I have to use a workaround?

Thanks in advance!
Coordinator
Feb 21, 2013 at 6:40 PM
Edited Feb 21, 2013 at 6:40 PM
The latest revision adds a new property (bool FtpClient.SocketKeepAlive) that allows you to set the KeepAlive flag on the underlying sockets. This may be of some use, I don't know; you'll have to test it and let me know. It's a new feature added just over a week ago. In regards to the IsConencted property, if the underlying socket is initalized its Connected property is returned, so you are right, m_socket, which is a System.Net.Sockets.Socket, doesn't know it's been disconnected. This is a common problem that can be worked around by testing the socket using Poll(), I thought it did this already but it may be broken; I'll look into it.

Aside from that, no there is no built-in way to do what you're asking with NOOP and CWD. System.Net.FtpClient just speaks the protocol according to RFC959 and some of the newer extensions. It's really something that should be implemented at a higher level than what this code provides. In a FTP client application, for example, you could have another thread watching and executing NOOP or CWD when the connection is idle. Such a feature won't be implemented in System.Net.FtpClient itself.
Coordinator
Feb 21, 2013 at 6:43 PM
By the way, if the polling is broken fixing it might address your problem, for example if the connection is reliably determined to be disconnected it will automatically (or should be) re-opened by the System.Net.FtpClient the time you call a method that executes a command.
Coordinator
Feb 21, 2013 at 7:44 PM
Revision 535ff1e690d1 adds a new property called SocketPollInterval that allows you to specify a period in miliseconds since the last activity before using Poll() to check if the socket is connected in the IsConnected property. This should yield more reliable detection of disconnected sockets. The default value is 15 seconds, so if 15 seconds have passed since the last read/write to the underlying stream the socket will first be polled and if it fails the stream will be cleaned up and false will be returned from IsConnected. The end result is that System.Net.FtpClient should gracefully re-connect and move on instead of an exception being thrown because it tried to read/write a disconnected socket. That last bit might take some tweaking, try it out and let me know how it works out for you.
Feb 22, 2013 at 12:13 AM
I tried the latest revision just at the moment, and it didn´t work, so I looked in the code and found the new if-statement.
I set a breakpoint and found out, that "m_socket.Poll(1000000, SelectMode.SelectRead)" is true, but "m_socket.Available" is not 0 therefore false. In my case it has the value 90, but I have no clue, what the client received, maybe a message from the server that its not connected or something like this (sadly the server log doesn´t tell anything after the disconnection). I also adressed the client to my network IP and tried to sniff the traffic, but my try was unsuccessful and there was just the traffic to other network clients/the internet.

After commenting the second part out it recognized correctly that the client was connected respectively disconnected. If I understood it correctly, the m_socket.Available should theoretically equal 0, because the client is disconnected and therefore it shouldn´t have received anything from the host. Could the connection check fail without "m_socket.Available" or did you add it just to double check it´s really disconnected?

Again thanks for your fast reaction!
Coordinator
Feb 22, 2013 at 1:25 AM
According to the MSDN, Poll() with SelectRead returns true if the connection has been disconnected or reset and true if there is data available, so in order to satisfy the test Poll() has to return true and the available data has to be 0 otherwise there is no way of knowing if the socket is disconnected short of attempting a read and catching the SocketException that would be thrown if it is disconnected. I'll dig into it some more tomorrow. It's weird that data is available unless the server is sending a message before the disconnection. Try sniffing the network with encryption disabled if you're not already just to see if we can figure out what's sitting on the socket that hasn't been read.

Also, did you by chance try the new SocketKeepAlive option? It's actually just a socket flag and may make no difference. I added it for a person that was using the code with embedded systems, haven't really tested it much on my end.
Coordinator
Feb 22, 2013 at 1:28 AM
When I test tomorrow I can setup my file zilla instance with a timeout (I didn't know it had this functionality until today...) and I'll see if I can reproduce the problem and debug some on my end.
Coordinator
Feb 22, 2013 at 1:33 AM
A temporary solution might also be to disable the thread safe data channels (FtpClient.EnableThreadSafeDataConnections) which actually clones your ftp client object and creates a new connection to the server leaving your primary control connection idle while a large transfer is taking place. I won't get into the specifics on why or how this works, just something worth trying especially if the connection is being lost while a transfer is in progress.
Feb 22, 2013 at 12:36 PM
Edited Feb 22, 2013 at 12:37 PM
Oh sorry, forgot to mention that i already have the SocketKeepAlive property since I use your library enabled. I tested it with a free ftp server host, the welcome message said "..kick after 3 min of inactivity..." but it didn´t. I´m not sure if it didn´t because of the SocketKeepAlive property or because the server hasn´t been set up correctly, because the FileZilla Server kicks me with the SocketKeepAlive property enabled.

I finally managed to connect to the FileZilla server on my computer (didn´t work without VPN, seeming it didn´t like the request from my ip to my ip) and I found out that the last message from the server was : "421 Connection timed out.\r\n" = 27 chars what is equal to the value of m_socket.Available = 27. After kicking the user the message was 29 chars long and m_socket.Available had the value 29. m_socket.Poll(...) was in every case true.

At the moment I´m just testing so it isn´t such a big problem, I just tested some special cases and noticed this, but for now I would use, as you suggested, another Task sending random commands for keeping the connnection alive.

For reproducing: I connected, listed a directory, kicked the user after success, waited around 20 sec to let the PollIntervall lapse away and tried to list the directory again. An IOException followed this in "FtpSocketStream" in the function "Read" at "ar = BaseStream.BeginRead" at line 388.
Coordinator
Feb 22, 2013 at 12:44 PM
Arlight, well I think one thing System.Net.FtpClient can do is check the Availalbe property of the socket and read off the last data still sitting there, then test for connectivity using Poll as described before. Technically speaking, if the poll interval has lapsed the Available property should be 0 because the connection has been idle so the fact that there is data on the socket that hasn't been read indicates something has gone awry. I've got some things to work on this morning and I'll think about how to address this problem.
Coordinator
Feb 22, 2013 at 1:21 PM
I just added some code to IsConnected that allows for a graceful detection and reconnection but I don't really like having the check there because if the SocketPollInterval property is set too low the test could yield false positives. I'm looking at some other possibilities but if you want to use this code for now, here it is (FtpSocketStream.cs):
public bool IsConnected {
            get {
                if (m_socket == null)
                    return false;

                if (!m_socket.Connected) {
                    Close();
                    return false;
                }

                if (m_socketPollInterval > 0 && DateTime.Now.Subtract(m_lastActivity).TotalMilliseconds > m_socketPollInterval) {
                    if (m_socket.Available > 0) {
                        byte[] buf = new byte[m_socket.Available];

#if DEBUG
                        Debug.WriteLine("Data is sitting on the socket and the poll interval has lapsed. I'm going to pull it off.");
#endif

                        m_socket.Receive(buf, buf.Length, 0);

#if DEBUG
                        Debug.WriteLine(string.Format("The data was: {0}", System.Text.Encoding.Default.GetString(buf).TrimEnd('\r', '\n')));
#endif
                    }

#if DEBUG
                    Debug.WriteLine("Poll()'ing sock for connectivity...");
#endif
                    if (m_socket.Poll(1000000, SelectMode.SelectRead) && m_socket.Available == 0) {
#if DEBUG
                        Debug.WriteLine("Looks like we've been disconnected...");
#endif
                        Close();
                        return false;
                    }
                }

                return true;
            }
        }
Coordinator
Feb 22, 2013 at 3:44 PM
Edited Feb 22, 2013 at 4:13 PM
The latest revision a6436a52b8c4 contains code that does essentially the same as what's posted above except it takes place in the Execute() method of FtpClient. There are some gotcha's. If the server closes the connection around the same time you attempt to perform a task, Poll() may not catch the interruption in the connection. There is no way that I know of around this. If you really want to handle the interruption gracefully you need to catch the IOException, look at the InnerException to see if it's a SocketException, and re-try the connection and method call again. Here's an example:
myMethod() {
  try {
     ftpconnection.doSomething();
  }
  catch(IOException e) {
    if(e.InnerException is SocketException) {
      myMethod();
    }
    else
      throw e;
  }
}
Feb 23, 2013 at 12:04 PM
OK, I will try to catch possible exceptions like this.

Thanks a lot for your fast support and solutions, I really appreciate that!
Mar 9, 2013 at 11:04 PM
Hi,

I tested again how the SocketPollInterval property is working and I found out, it isn´t.
I think the source of error is in this line in the file "FtpSocketStream.cs" at line ~160 :
if (m_socket.Poll(1000000000, SelectMode.SelectRead) && m_socket.Available == 0) {...
According to MSDN its because of the number 1 000 000 000 µs = 1000 seconds.
MSDN says : "Poll will block execution until the specified time period, measured in microseconds, elapses.". And thats exactly what happens, it wants to wait 1000 seconds, but an exception appears after a short amount of time. After setting the value to 10 seconds it waited 10 seconds after the SocketPollInterval elapsed and continued then as usual.

In the version a few posts above this one the timespan was just 1 second. I don´t know what exactly Poll does (whether it affects the connection to the ftp-server or just the local client) but maybe you have an idea of a matching timespan for this.
Coordinator
Mar 13, 2013 at 3:56 PM
I'll look into this problem soon, I've been extremely busy with other work so I haven't had a chance yet. Just wanted to let you know I'm not ignoring it.
Coordinator
Mar 18, 2013 at 6:41 PM
I've reduced the time to 500 milliseconds in the latest revision. I'm not sure of the optimal time so we'll just have to try and see.
Mar 18, 2013 at 10:58 PM
OK, thanks a lot!