DirectoryExists causes a FtpCommandException

Aug 21, 2012 at 4:40 PM

I'm connecting to a Microsoft FTP server, and the first thing I'm doing is checking to see if a directory exists, and if not, create it.  No matter what form I specify for the folder ("test", "/test", or "test/"), it's generating the exception.  I'm using FtpClient.DirectoryExists().


Log and stack trace:

> 220-Microsoft FTP Service
> 220 ed2c Web FTP
< AUTH TLS
> 500 'AUTH TLS': command not understood
< AUTH SSL
> 500 'AUTH SSL': command not understood
< USER backup
> 331 Password required for backup.
< PASS [omitted for security]
> 230-Welcome to the FTP server!
> 230 User backup logged in.
< FEAT
> 211-FEAT
>     SIZE
>     MDTM
> 211 END
< TYPE A
> 200 Type set to A.
< EPSV
> 500 'EPSV': command not understood
< PASV
> 227 Entering Passive Mode (99,190,188,74,192,137).
< LIST /test
> 125 Data connection already open; Transfer starting.
> 550 /test: The system cannot find the file specified. 

System.Net.FtpClient.FtpCommandException: /test: The system cannot find the file specified. 
   at System.Net.FtpClient.FtpDataStream.Close() in C:\Work\SourceControl\ClassLib\Third-Party\Web\System.Net.FtpClient\trunk\FtpDataStream.cs:line 507

response type:
	System.Net.FtpClient.FtpResponseType.PermanentNegativeCompletion
completion code:
	550

Any help is much appreciated!

Coordinator
Aug 21, 2012 at 8:55 PM

I'm looking into this now, seems to be a bug in the GetObjectInfo() code with the LIST command. Will post back when I have it fixed.

Coordinator
Aug 21, 2012 at 9:31 PM

I've just uploaded a new revision of the 2012.08.08 branch that should address this issue. I tested it against IIS on server 2008R2. The revision is cc2adcf7cbc5. Let me know if it doesn't take care of the problem.

Thanks

Coordinator
Aug 21, 2012 at 9:36 PM

Just committed the same fix for the default branch if that's what you're running. The revision is bf99e6af7277.

Aug 21, 2012 at 10:15 PM

Hello,

I am also running into this issue, the fix seems to solve the problem in the function DirectoryExists, but the same error seems to occur in CreateDirectory as well.

Coordinator
Aug 22, 2012 at 12:14 AM

CreateDirectory calls the MKD command on the server. If the parent directories of the directory do not exist I would expect it to return a 5xx reply which would cause a FtpCommandException to be thrown by the CreateDirectory() method. If the FTP MKD command doesn't create preceding directories in the path then CreateDirectory will not either. If you want this functionality in a portable way you need to check that all directories in the path exist before hand. Here is quick, dirty and untested example of how I would go about doing this:

void CreateDirectoryRecursive(FtpClient ftpClient, string path) {
    if(!ftpClient.DirectoryExists(System.IO.Path.GetDirectoryName(path)) {
    	CreateDirectoryRecursive(System.IO.Path.GetDirectoryName(path));
	}

	ftpClient.CreateDirectory(path);
}

Coordinator
Aug 22, 2012 at 12:44 AM

Worth mentioning that the root cause of the issue that spawned this thread was that when a directory does not exist a FtpCommandException() is thrown because the server sends a 5xx  permanent failure reply in response to the LIST command. The FTP protocol doesn't provide any straight forward facility to test if a file or directory exists. The way I implemented it is to try to get a listing of the parent directory (there was a bug getting the parent directory too) and see if the the sub directory is in the listing. If you called DirectoryExists("/none/of/this/path/exists"), in the case of IIS, LIST is called on "/none/of/this/path" to see if "exists" comes up as an object in the directory.  If "/none/of/this/path" doesn't exist on the server the server sends a 5xx permanent failure reply.

The work around is to catch the FtpCommandException() that's thrown and assume the directory doesn't exist because LISTing the parent directory triggered a failure. Does this really mean the directory doesn't exist? That question is very difficult to answer. What if the 5xx failure was caused because the server is malfunctioning or you don't have permission to the list the contents of "/none/of/this/path"? There is no way through the FTP protocol to determine why it failed that I'm aware of. At best you can examine the server's reply message but trying to write code to interpret the actual message is doomed to failure because the message from server x could and almost certainly will be completely different than server y's 5xx response under the same circumstances.

Anyway, I felt it was worth elaborating on why it failed and explain why these seemingly simple methods are very difficult to implement. None of this has to do with the CreateDirectory() method which uses non of the code in regards to the original posters problem. CreateDirectory takes a path parameter executes "MKD /the/path/you/gave" on the server and if a 4xx or 5xx failure reply are sent from the server it's because there is either a problem with the path you supplied or the server is malfunctioning, in other words it's not a bug in System.Net.FtpClient.

Aug 22, 2012 at 3:34 AM

Thanks for your reply, yes you were correct the same server that caused the exception with getting the directory listing also could not create a directory structure. When testing with filezilla server this worked without any issue but on our cdn it acted differently. Your solution to create the directories recursively worked like a charm, but actually there is no need to do this function recursively. Here is what I am using:

 

        /// <summary>
        /// CreateDirectoryRecursive
        /// </summary>
        /// <param name="cl">FTP client: System.Net.FtpClient.FtpClient</param>
        /// <param name="path">Path on the remote FTP server, excluding host name.</param>
        public static void CreateDirectoryRecursive(ref FtpClient cl, string path)
        {

           //init
           string[] pathArray = path.Split(Convert.ToChar("/"));
           string ascendingPath = "";

            //loop through path segments to create foleders
           foreach (string folder in pathArray)
            {

               //verify string is not empty (first case if path starts with /)
                if (!string.IsNullOrEmpty(folder))
                {
                    //build acsending path
                    ascendingPath += "/" + folder;
                    //check if directory exists at this level and if not create it.
                    if (!cl.DirectoryExists(ascendingPath)) { cl.CreateDirectory(ascendingPath); }
                }
            }
        }

Coordinator
Aug 22, 2012 at 11:40 AM

This works too however keep in mind that DirectoryExists() and FileExists() trigger a file listing which means a data connection is opened every time you make a call. If you go through the array indexes in a descending order you can and probably will reduce the number of data connections that need to be opened to the server.

Aug 22, 2012 at 4:24 PM

Thank you, jptrosclair, for the fast response and code change to resolve the original DirectoryExists() issue.  I haven't had a chance to try it out myself, but I'm sure it'll be fine.

jason, in addition to jp's comments, you also need to be aware that, in your version of CreateDirectoryRecursive(), the first iteration will prepend a "/" to the first folder, regardless of whether or not the original "path" value begins with a "/".  Specifying this absolute path may or may not be what you always want.  I just wanted to point that out in case it is of any concern to you.
Also, you may want to rename the method if you're avoiding recursion for some reason :-) 

Aug 22, 2012 at 8:56 PM

Ran into a small bug affecting at least the latest revision.  On line 67 of FtpPassiveStream.cs, you need to change:

this.ControlConnection.Execute("PASV"); 

to

reply = this.ControlConnection.Execute("PASV");

If you'd prefer that I create items in the Issue Tracker for things such as this in the future, rather than posting in a discussion, just let me know.

Thanks again!

Coordinator
Aug 23, 2012 at 2:20 PM

Thanks, fixed in the latest revision

Developer
Sep 11, 2012 at 5:59 PM
Edited Sep 11, 2012 at 6:02 PM

Hi, I've just checked the last version of the code.

With the DirectoryExists and FileExists fixes that uses Path.GetDirectoryName and Path.GetFileName it is impossible to call GetObjectInfo on "/".
I think that's why it was does by hand before the fix.

It crashes because "/" cannot be resolved byt Path.GetDirectoryName since it works with "\\" (windows format).

I don't think this is the excepted behavior.

Coordinator
Sep 13, 2012 at 12:30 AM

I'll look into that, but also GetObjectInfo() on the root using the LIST command won't work anyway for this scenario because it will return a directory listing of /. I have some new code I've been working and in order to implement a DirectoryExists() method I've decided that attempting CWD to that directory and back to the old working directory is a much better alternative not only because it doesn't have the problem I mentioned above but also because it doesn't require opening a data connection which is always a good thing in terms of performance. I think I'm going to change the DirectoryExists implementation in 2012.08.08 and the default branches to use this method instead of a file listing.

Coordinator
Sep 13, 2012 at 2:35 PM

Committed a new revision to the default branch this morning that has a new DirectoryExists() implementation using CWD as well as several bug fixes. I won't be back porting any of this to the 2012.08.08 branch, will try to make a new release soon off the default branch.