This project is read-only.

How do an Async Download

Apr 25, 2014 at 5:50 PM
I have some cheap hosting that suits my needs quite well but of course with cheap hosting they could disappear one day(though they been in business for a number of years) so I figured that I could just upload my backups to blob storage what is pretty cheap.

My plan is to have a quartz job that runs and connects to my ftp on my hosting site and downloads the file and then uploads it into my blob container.

I would like to do this async and it seems like System.Net.FtpClient has this option but I am confused on how to use it.

I found this in their examples but confused by it.
public static class BeginOpenReadExample {
        static ManualResetEvent m_reset = new ManualResetEvent(false);

        public static void BeginOpenRead() {
            // The using statement here is OK _only_ because m_reset.WaitOne()
            // causes the code to block until the async process finishes, otherwise
            // the connection object would be disposed early. In practice, you
            // typically would not wrap the following code with a using statement.
            using (FtpClient conn = new FtpClient()) {
                m_reset.Reset();

                conn.Host = "localhost";
                conn.Credentials = new NetworkCredential("ftptest", "ftptest");
                conn.BeginOpenRead("/path/to/file",
                    new AsyncCallback(BeginOpenReadCallback), conn);

                m_reset.WaitOne();
                conn.Disconnect();
            }
        }

        static void BeginOpenReadCallback(IAsyncResult ar) {
            FtpClient conn = ar.AsyncState as FtpClient;

            try {
                if (conn == null)
                    throw new InvalidOperationException("The FtpControlConnection object is null!");

                using (Stream istream = conn.EndOpenRead(ar)) {
                    byte[] buf = new byte[8192];

                    try {
                        DateTime start = DateTime.Now;

                        while (istream.Read(buf, 0, buf.Length) > 0) {
                            double perc = 0;

                            if (istream.Length > 0)
                                perc = (double)istream.Position / (double)istream.Length;

                            Console.Write("\rTransferring: {0}/{1} {2}/s {3:p}         ",
                                          istream.Position.FormatBytes(),
                                          istream.Length.FormatBytes(),
                                          (istream.Position / DateTime.Now.Subtract(start).TotalSeconds).FormatBytes(),
                                          perc);
                        }
                    }
                    finally {
                        Console.WriteLine();
                        istream.Close();
                    }
                }
            }
            catch (Exception ex) {
                Console.WriteLine(ex.ToString());
            }
            finally {
                m_reset.Set();
            }
        }
    }
the "BeginOpenReadCallback" area confuses me, I don't get why they call "EndOpenRead" read right away(thought that would be after the stream is read).

It also seems like they are not doing much with the read, I am not sure what I should store stream data after the read in.

I see that azure blob sdk has this
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
    CloudConfigurationManager.GetSetting("StorageConnectionString"));

// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");

// Retrieve reference to a blob named "myblob".
CloudBlockBlob blockBlob = container.GetBlockBlobReference("myblob");

// Create or overwrite the "myblob" blob with contents from a local file.
using (var fileStream = System.IO.File.OpenRead(@"path\myfile"))
{
    blockBlob.UploadFromStream(fileStream);
} 
so not sure if I could just merge the ftp read with this UploadFromStream method.