Issues with Explicit FTP over TLS

Feb 27, 2014 at 6:13 PM
Hi, I'm facing an issue while trying to use Explicit FTP over TLS.

The test server is FileZilla (v.0.9.43) with marked following settings:
Allow explicit FTP over TLS,
Disallow plain unencrypted FTP,
Force PROT P to encrypt file transfers in SSL/TLS mode

Here is an example of the code that fails:
var ftpClient = new FtpClient
                {
                    Host = "localhost",
                    Port = 21,
                    EncryptionMode = FtpEncryptionMode.Explicit,
                    SocketKeepAlive = false,
                    DataConnectionType = FtpDataConnectionType.PASV,
                    Credentials = new NetworkCredential("test", "Test123$"),
                    DataConnectionEncryption = true
                };

ftpClient.ValidateCertificate += (control, e) => {e.Accept = true;};
ftpClient.Connect();

var items = ftpClient.GetListing("/inbox");

var result = (from item in items select item.Name).ToArray();

ftpClient.Disconnect();

result.ToList().ForEach(Console.WriteLine);
Executing the above code gives following exception:
System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> System.ComponentModel.Win32Exception: The message received was unexpected or badly formatted
--- End of inner exception stack trace ---
at System.Net.FtpClient.FtpSocketStream.ActivateEncryption(String targethost, X509CertificateCollection clientCerts)
at System.Net.FtpClient.FtpClient.OpenPassiveDataStream(FtpDataConnectionType type, String command, Int64 restart)
at System.Net.FtpClient.FtpClient.OpenDataStream(String command, Int64 restart)
at System.Net.FtpClient.FtpClient.GetListing(String path, FtpListOption options)
at System.Net.FtpClient.FtpClient.GetListing(String path)
And this is the FileZila server log:
(000015)2/27/2014 21:01:56 PM - (not logged in) (::1)> Connected, sending welcome message...
(000015)2/27/2014 21:01:56 PM - (not logged in) (::1)> AUTH TLS
(000015)2/27/2014 21:01:57 PM - (not logged in) (::1)> 234 Using authentication type TLS
(000015)2/27/2014 21:01:57 PM - (not logged in) (::1)> SSL connection established
(000015)2/27/2014 21:01:57 PM - (not logged in) (::1)> USER test
(000015)2/27/2014 21:01:57 PM - (not logged in) (::1)> 331 Password required for test
(000015)2/27/2014 21:01:57 PM - (not logged in) (::1)> PASS ********
(000015)2/27/2014 21:01:57 PM - test (::1)> 230 Logged on
(000015)2/27/2014 21:01:57 PM - test (::1)> PBSZ 0
(000015)2/27/2014 21:01:57 PM - test (::1)> 200 PBSZ=0
(000015)2/27/2014 21:01:57 PM - test (::1)> PROT P
(000015)2/27/2014 21:01:57 PM - test (::1)> 200 Protection level set to P
(000015)2/27/2014 21:01:57 PM - test (::1)> FEAT
(000015)2/27/2014 21:01:57 PM - test (::1)> 211-Features:
(000015)2/27/2014 21:01:57 PM - test (::1)> MDTM
(000015)2/27/2014 21:01:57 PM - test (::1)> REST STREAM
(000015)2/27/2014 21:01:57 PM - test (::1)> SIZE
(000015)2/27/2014 21:01:57 PM - test (::1)> MLST type*;size*;modify*;
(000015)2/27/2014 21:01:57 PM - test (::1)> MLSD
(000015)2/27/2014 21:01:57 PM - test (::1)> AUTH SSL
(000015)2/27/2014 21:01:57 PM - test (::1)> AUTH TLS
(000015)2/27/2014 21:01:57 PM - test (::1)> PROT
(000015)2/27/2014 21:01:57 PM - test (::1)> PBSZ
(000015)2/27/2014 21:01:57 PM - test (::1)> UTF8
(000015)2/27/2014 21:01:57 PM - test (::1)> CLNT
(000015)2/27/2014 21:01:57 PM - test (::1)> MFMT
(000015)2/27/2014 21:01:57 PM - test (::1)> 211 End
(000015)2/27/2014 21:01:57 PM - test (::1)> OPTS UTF8 ON
(000015)2/27/2014 21:01:57 PM - test (::1)> 202 UTF8 mode is always enabled. No need to send this command.
(000015)2/27/2014 21:01:57 PM - test (::1)> CWD /inbox
(000015)2/27/2014 21:01:57 PM - test (::1)> 250 CWD successful. "/inbox" is current directory.
(000015)2/27/2014 21:01:57 PM - test (::1)> PWD
(000015)2/27/2014 21:01:57 PM - test (::1)> 257 "/inbox" is current directory.
(000015)2/27/2014 21:01:57 PM - test (::1)> TYPE I
(000015)2/27/2014 21:01:57 PM - test (::1)> 200 Type set to I
(000015)2/27/2014 21:01:57 PM - test (::1)> EPSV
(000015)2/27/2014 21:01:57 PM - test (::1)> 229 Entering Extended Passive Mode (|||8471|)
(000015)2/27/2014 21:01:57 PM - test (::1)> MLSD /inbox
(000015)2/27/2014 21:01:57 PM - test (::1)> 150 Opening data channel for directory listing of "/inbox"
(000015)2/27/2014 21:01:57 PM - test (::1)> 426 Connection closed; aborted transfer of "/inbox"
(000015)2/27/2014 21:01:58 PM - test (::1)> disconnected.
This exception is also thrown if I try to upload a file while EnableThreadSafeDataConnections is set to false. If it's set to true (default) the file upload is successful.


Any advice on how solve the issue?
Feb 28, 2014 at 4:26 PM
Ok, so I've went around the issue by modifying the GetListing method in FtpClient.cs @line 1867 as follows:
try {
                m_lock.WaitOne();

                FtpClient client = null;
                FtpDataStream stream = null;                        

                if (m_threadSafeDataChannels) {
                    client = CloneConnection();
                    client.Connect();
                    client.SetWorkingDirectory(GetWorkingDirectory());
                }
                else {
                    client = this;
                }

                client.SetDataType(FtpDataType.Binary);                

                // read in raw file listing
                using (stream = client.OpenDataStream(string.Format("{0} {1}", listcmd, path.GetFtpPath()), 0))
                {
                    try {
                        while ((buf = stream.ReadLine(Encoding)) != null) {
                            if (buf.Length > 0) {
                                rawlisting.Add(buf);
                                FtpTrace.WriteLine(buf);
                            }
                        }
                    }
                    finally {
                        stream.Close();
                    }
                }
            }
            finally {
                m_lock.ReleaseMutex();
            }
For some odd reason the "cloned" connection works as expected.

Adding the FtpTrace output:
InterNetworkV6: ::1
220 FileZilla Server version 0.9.43 beta
AUTH TLS
234 Using authentication type TLS
Time to activate encryption: 0h 0m 0s, Total Seconds: 0.035907.
USER test
331 Password required for test
PASS <omitted>
230 Logged on
PBSZ 0
200 PBSZ=0
PROT P
200 Protection level set to P
FEAT
211-Features:
 MDTM
 REST STREAM
 SIZE
 MLST type*;size*;modify*;
 MLSD
 AUTH SSL
 AUTH TLS
 PROT
 PBSZ
 UTF8
 CLNT
 MFMT
211 End
Text encoding: System.Text.SBCSCodePageEncoding
CWD /inbox
250 CWD successful. "/inbox" is current directory.
PWD
257 "/inbox" is current directory.
TYPE I
200 Type set to I
Changed data connection type to EPSV because we are connected with IPv6.
EPSV
229 Entering Extended Passive Mode (|||18156|)
InterNetworkV6: ::1
MLSD /inbox
150 Opening data channel for directory listing of "/inbox"
Disposing FtpClient object...
There is stale data on the socket, maybe our connection timed out. Re-connecting.
Not sending QUIT because the connection has already been closed.
Disposing FtpSocketStream...
System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> System.ComponentModel.Win32Exception: The message received was unexpected or badly formatted
   --- End of inner exception stack trace ---
   at System.Net.FtpClient.FtpSocketStream.ActivateEncryption(String targethost, X509CertificateCollection clientCerts) in c:\Debug\System.Net.FtpClient\FtpSocketStream.cs:line 678
   at System.Net.FtpClient.FtpClient.OpenPassiveDataStream(FtpDataConnectionType type, String command, Int64 restart) in c:\Debug\System.Net.FtpClient\FtpClient.cs:line 1125
   at System.Net.FtpClient.FtpClient.OpenDataStream(String command, Int64 restart) in c:\Debug\System.Net.FtpClient\FtpClient.cs:line 1264
   at System.Net.FtpClient.FtpClient.GetListing(String path, FtpListOption options) in c:\Debug\System.Net.FtpClient\FtpClient.cs:line 1886
   at System.Net.FtpClient.FtpClient.GetListing(String path) in c:\Debug\System.Net.FtpClient\FtpClient.cs:line 1798
   at Tests.Program.Test() in c:\Debug\Tests\Program.cs:line 107
   at Tests.Program.Main(String[] args) in c:\Debug\Tests\Program.cs:line 29
--DONE--
Coordinator
Mar 3, 2014 at 9:24 PM
This line from your transaction:
There is stale data on the socket, maybe our connection timed out. Re-connecting.
Not sending QUIT because the connection has already been closed.
Indicates that the server is sending something unexpected and closing the connection. If you disable encryption what it sent should be logged to your trance listeners. Usually this means the connection is timing out. It could happen on a large file listing but it shouldn't; the server should never "timeout" the control connection while a data connection is in use. Some servers do indeed do this though. Anyway, it's hard to say what's really going on without seeing what the 'stale data' that's sitting on the socket is.
Mar 6, 2014 at 10:07 AM
This is the ftp-trace w/o encryption:
cl.EncryptionMode = FtpEncryptionMode.None;
cl.EnableThreadSafeDataConnections = false;
TYPE I
200 Type set to I
Changed data connection type to EPSV because we are connected with IPv6.
EPSV
229 Entering Extended Passive Mode (|||50097|)
InterNetworkV6: ::1
MLSD /inbox
150 Opening data channel for directory listing of "/inbox"
type=file;modify=20131112162330;size=5120; 20111222A00016195446
type=file;modify=20131112162122;size=298409; 20120208A00016597946
type=file;modify=20131112162153;size=611328; 20120222A00016709151
226 Successfully transferred "/inbox"
Disposing FtpSocketStream...
20111222A00016195446
20120208A00016597946
20120222A00016709151
/inbox/20111222A00016195446
/inbox/20120208A00016597946
/inbox/20120222A00016709151
QUIT
221 Goodbye
Disposing FtpClient object...
Disposing FtpSocketStream...
--DONE--
The issue appears only when encryption is enabled and the Thread Safe Data Connections are disabled (cl.EnableThreadSafeDataConnections = false;).
Tested on few different FTP servers both Linux and Windows based.

Steps to reproduce (for me following steps worked on Win 7 Pro, Win 8.1 Pro, Server 2012):
1) Download latest version of FileZilla Server
2) Install with default settings
3) Create test user and set it with full access to some folder
4) In FileZilla Server Options under the SSL/TLS section set as follows:
4.a) Generate a test self signed certificate using the "Generate new Certificate..." button
4.b) Mark "Enable FTP over SSL/TLS support (FTPS)"
4.c) Mark "Allow explicit FTP over TLS"
4.d) Mark "Disallow plain unencrypted FTP"
4.e) Mark "Force PROT P to encrypt file transfers in SSL/TLS mode"
5) In the test code make sure to set:
5.a) cl.EncryptionMode = FtpEncryptionMode.Explicit;
5.b) cl.EnableThreadSafeDataConnections = false;

I'm really puzzled with the issue, which appears to be a problem with the SSL handshake,
as tracing the code shows that both normal and cloned control connections are opening the FTP data stream in the same way.
Coordinator
Mar 9, 2014 at 4:52 AM
Wonder if it's a problem with that particular version of FileZilla? I've tested heavily against FileZilla server, intact it is my goto dev server on windows, but I've yet to see any issues like this while working with it. I have seen that particular exception thrown when using invalid client certificates with SslStream (covered on the documentation page) in System.Net.FtpClient but that's not the case here.
May 19, 2014 at 8:32 PM
I am having the same problem, an SSPI error when listing a directory. This is the "latest" version of FileZilla server. Was there a solution? I have tried both states for EnableThreadSafeDataConnections with no luck.

Stack trace:
System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> System.ComponentModel.Win32Exception: The message received was unexpected or badly formatted
   --- End of inner exception stack trace ---
   at System.Net.FtpClient.FtpSocketStream.ActivateEncryption(String targethost, X509CertificateCollection clientCerts)
   at System.Net.FtpClient.FtpClient.OpenPassiveDataStream(FtpDataConnectionType type, String command, Int64 restart)
   at System.Net.FtpClient.FtpClient.OpenDataStream(String command, Int64 restart)
   at System.Net.FtpClient.FtpClient.GetListing(String path, FtpListOption options)
The configuration code block:
protected FtpClient InitializeFtpClient() {
    var ftpClient = new FtpClient();
    ftpClient.Credentials = new System.Net.NetworkCredential(FtpAccount, FtpPassword);
    ftpClient.DataConnectionType = FtpDataConnectionType.AutoPassive;
    ftpClient.DataConnectionEncryption = true;
    ftpClient.DataConnectionConnectTimeout = 3000000;
    ftpClient.DataConnectionReadTimeout = 3000000;
    ftpClient.ConnectTimeout = 3000000;
    ftpClient.ReadTimeout = 3000000;
    ftpClient.SocketKeepAlive = false;
    ftpClient.SocketPollInterval = 0;
    ftpClient.Host = FtpHost;
    ftpClient.Port = FtpPort;
    if (null != FtpCertificate) {
        // We assume 2-way SSL
        // Per this link there appears to be a problem with the ftp library and when encrypt is enabled while threadsafe is false.
        // http://netftp.codeplex.com/discussions/535815
        //TODO: this works for FileZilla - Humana too?
        ftpClient.EncryptionMode = FtpEncryptionMode.Explicit;
        ftpClient.EnableThreadSafeDataConnections = false;
        ftpClient.ClientCertificates.Add(FtpCertificate);
        ftpClient.ValidateCertificate += (control, e) => { e.Accept = true; };
        return ftpClient;
    }
    // else 1-way SSL
    ftpClient.EncryptionMode = FtpEncryptionMode.Implicit;
    ftpClient.EnableThreadSafeDataConnections = false;
    return ftpClient;
}
And the call that fails - the code successfully authenticates over explicit, and I'm able to query and set the working directory. It is the GetListing that causes the exception.
using (var ftpClient = InitializeFtpClient()) {
try {
    Connect();
} catch (Exception ex) {
    CleanupEarlyExit(ex, null, String.Format("Failed FTP connection server [{0}] account [{1}], exiting", ftpClient.Host, FtpAccount));
    return;
}
try {
    var dir = ftpClient.GetWorkingDirectory();
} catch (SocketException sockEx) {
    CleanupEarlyExit(sockEx, null, "Failed to get working directory");
    return;
} catch (Exception ex) {
    CleanupEarlyExit(ex, null, "Failed to get working directory");
    return;
}
try {
    var exists = ftpClient.DirectoryExists(FtpSourceFolder);
} catch (Exception ex) {
    CleanupEarlyExit(ex, null, "Failed testing source folder");
    return;
}
try {
    ftpClient.SetWorkingDirectory(FtpSourceFolder);
    Display(String.Format("...... working folder set [{0}]", FtpSourceFolder));
} catch (Exception ex) {
    CleanupEarlyExit(ex, null, "Failed setting working folder");
    return;
}
try {
__  var allFiles = ftpClient.GetListing(FtpSourceFolder, FtpListOption.ForceList | FtpListOption.AllFiles | FtpListOption.Modify);__
/*****/
Coordinator
May 19, 2014 at 9:08 PM
Check the docs page about using client certificates and SSPI exception.
May 19, 2014 at 11:50 PM
To make sure I understand, a pem certificate will pass explicit AUTH/TLS, can roundtrip to get the working directory, and then and only then fail on a getdirectory?

Not trying to be tedious. I have to go back and explain to a federal contractor who has been generating rooted certs for some years that they are wrong and you are right. :)
May 19, 2014 at 11:53 PM
Also, I expect I'd run into the same consideration with someone else's library. Can you give me some background on why this is a problem so I'll know what to look for and the right questions to ask?
Coordinator
May 20, 2014 at 12:21 AM
I don't know if other libraries have this problem or not; System.Net.FtpClient uses .net's SslStream class, there is no code written here that handles SSL directly. What I mean to say is the problem with .pem certificates is not a problem of System.Net.FtpClient but rather a problem of SslStream which is included in the framework. As to the difference, pem certificates typically don't include the signing certificate and the private key where p12 certificates do.

Types of certificates and differences
Why don't OpenSSL certificates work with SslStream?
SslStream client unable to complete handshake
SslStream failed to authenticate as client
Thread here where the certificate format problem was first brought to my attention

If you want more evidence than that talk to Microsoft, as I mentioned before System.Net.FtpClient uses the native SSL support in .net.
Coordinator
May 20, 2014 at 12:29 AM
davescheffer wrote:
To make sure I understand, a pem certificate will pass explicit AUTH/TLS, can roundtrip to get the working directory, and then and only then fail on a getdirectory?
I'm not sure I really understand the problem you're describing here. Client certificates are only a requirement if a server is configured to require clients to have them. If the server doesn't have this requirement then it's not necessary to specify a client certificate to use encryption.
May 20, 2014 at 12:43 AM
Ok I think I am good to go on next steps. I really appreciate the immediate replies with great links. Thanks.
Coordinator
May 20, 2014 at 1:24 AM
Not a problem, good luck.