How can I improve the response time of GetListing()

May 8, 2014 at 12:00 PM
Hi,
I am using this function to return several hundred rows of file info from an ftp server and adding them to a collection I can present in a grid in an MVC application. The trouble is, it takes about ten minutes to return the collection. Any ideas what I could be doing to improve the speed in which the collection is returned?

private bool Connect()
    {
        try
        {
            _conn.DataConnectionType = FtpDataConnectionType.AutoPassive;
            _conn.Host = _host;
            _conn.Port = _port;
            _conn.Credentials = new NetworkCredential(_user, _pass);
            _conn.Connect();
        catch (Exception ex)
        {
            LogHandler.LogMessage(ex);
        }
}

public IEnumerable<FtpFile> GetFileListing()
    {
        try
        {
            if (Connect())
            {
                _ftpFileList.Clear();

                var list = _conn.GetListing(_conn.GetWorkingDirectory(), FtpListOption.Modify | FtpListOption.Size);

                foreach (FtpListItem item in list)
                {
                    if (item.Type.Equals(FtpFileSystemObjectType.File))
                    {
                        FtpFile file = new FtpFile();
                        file.Created = item.Created;
                        file.FullName = item.FullName;
                        file.GroupPermissions = item.GroupPermissions;
                        file.Input = item.Input;
                        file.LinkObject = item.LinkObject;
                        file.LinkTarget = item.LinkTarget;
                        file.Modified = item.Modified;
                        file.Name = item.Name;
                        file.OthersPermissions = item.OthersPermissions;
                        file.OwnerPermissions = item.OwnerPermissions;
                        file.Size = item.Size;
                        file.SpecialPermissions = item.SpecialPermissions;
                        file.Type = item.Type;
                        _ftpFileList.Add(file);
                    }
                }
            }
        }
        catch (Exception ex)
        {
            LogHandler.LogMessage(ex);
        }
        finally
        {
            Disconnect();
        }
        return _ftpFileList;
    }
Coordinator
May 8, 2014 at 1:42 PM
Read the docs about file listings. If you're connecting to a server that doesn't support MLSD (Machine Listings) System.Net.FtpClient may very well be sending multiple requests to the server for the Modified Date and File Size. Enable transaction logging with FtpTrace (to a file since this is a MVC app) and post it here so we can see exactly what System.Net.FtpClient is doing.

Bottom line is with that many files anything other than MLSD is most likely going to be slow; 10 minutes seems awfully long even so but it hard to make any accurate observation as to why without seeing the transaction log.
Coordinator
May 8, 2014 at 2:08 PM
Also, in your code example you're copying the FtpList item over to new FtpList items and adding them to a List<T>. GetListing() returns FtpListItem[], there's no reason to copy them to new objects unless I'm missing something with your code. Unless you need them in a collection there's really no reason to iterate over them there at all. Just an observation.
May 13, 2014 at 4:01 PM
The server does not support MLSD.
I am copying to List<FtpFile> which is a model I have created for the page with a similar name, sorry for the confusion.

Here is the trace, I changed some sensitive data like servername, user, files, but the rest is what came back:

220 <<<IBM Sterling Connect:Enterprise for UNIX 2.5.00 Secure FTP>>> at serverName FTP server ready. Time = 10:48:34
USER user
331 Password required for user.
PASS <omitted>
230 IBM Sterling Connect:Enterprise for UNIX login ok, access restrictions apply.
FEAT
500 'FEAT': command not understood.
Text encoding: System.Text.ASCIIEncoding
SYST
215 UNKNOWN Type: L8
PWD
257 "/user" is current directory.
PWD
257 "/user" is current directory.
TYPE I
200 Type set to I.
EPSV
500 'EPSV': command not understood.
PASV
227 Entering Passive Mode (209,191,158,210,39,41)
LIST /user
150 Opening BINARY mode data connection for /user.
-SR--M------- A user 387322 120506 Aug 27 2013 <<ACTIVITY LOG>>
-CR-------FTS A user 312559 0 May 08 11:52 fileName.dat
Total number of batches listed: 184
226 Transfer complete.
Disposing FtpSocketStream...
Failed to parse file listing: -SR--M------- A user 387322 120506 Aug 27 2013 <<ACTIVITY LOG>>
Failed to parse file listing: Total number of batches listed: 184
QUIT
221 Goodbye.
May 13, 2014 at 6:28 PM
Also, I have changed the FtpDataConnectionType to PASV which got rid of the error:
EPSV
500 'EPSV': command not understood.
Coordinator
May 14, 2014 at 3:28 PM
Judging by what I see there, nothing short of a faster connection and/or computer.