Connection closed by the Remotehost

Apr 3, 2013 at 1:30 PM
Edited Apr 3, 2013 at 1:36 PM
Hey,

With this code:
 For Each folder In listLocalDirs
                If ftpclient.DirectoryExists(folder.FullName.Replace(sLocalDir, sRemoteDir).Replace("\", "/")) = False Then
                    Thread.Sleep(100)
                    ftpclient.CreateDirectory(folder.FullName.Replace(sLocalDir, sRemoteDir).Replace("\", "/"))
                End If
            Next
With very complex folders I run into a IOException ("Von der Übertragungsverbindung können keine Daten gelesen werden: Eine vorhandene Verbindung wurde vom Remotehost geschlossen.") that means that he cannot read data from the connection, because the connection was canceled by the remotehost. With Thread.Sleep I managed to create more directories before the error occurs. How can I dodge this error?

Log from the last elements before the exception occurs:
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 11
550 CWD failed. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 11": directory not found.
CreateDirectory("BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 11", True)
MKD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 11
257 "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 11" created successfully
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 12
550 CWD failed. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 12": directory not found.
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY
250 CWD successful. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY" is current directory.
CWD /
250 CWD successful. "/" is current directory.
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 12
550 CWD failed. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 12": directory not found.
CreateDirectory("BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 12", True)
MKD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 12
257 "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 12" created successfully
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 2
550 CWD failed. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 2": directory not found.
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY
250 CWD successful. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY" is current directory.
CWD /
250 CWD successful. "/" is current directory.
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 2
550 CWD failed. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 2": directory not found.
CreateDirectory("BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 2", True)
MKD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 2
257 "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 2" created successfully
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 3
550 CWD failed. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 3": directory not found.
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY
250 CWD successful. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY" is current directory.
CWD /
250 CWD successful. "/" is current directory.
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 3
550 CWD failed. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 3": directory not found.
CreateDirectory("BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 3", True)
MKD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 3
257 "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 3" created successfully
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 4
550 CWD failed. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 4": directory not found.
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY
250 CWD successful. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY" is current directory.
CWD /
250 CWD successful. "/" is current directory.
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 4
550 CWD failed. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 4": directory not found.
CreateDirectory("BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 4", True)
MKD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 4
257 "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 4" created successfully
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 5
550 CWD failed. "/BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY/CPU 5": directory not found.
PWD
257 "/" is current directory.
CWD BuildYourCloud2/Rainmeter/Skins/BlueVision V0.2/CPU/CPU 1-12 GRAPH ONLY
Eine Ausnahme (erste Chance) des Typs "System.IO.IOException" ist in System.dll aufgetreten.
No matter what, it always crashes on the CPU 5 folder. Sometimes the exception says that there are invalid characters in the path.
It also crashes random, when the FTP server cancels the connection. I can't see any disconnecting on the server log (FileZilla on localhost).
Coordinator
Apr 3, 2013 at 3:50 PM
I've seen issues like this with SslStream when running with batch processes. I don't know of a solution other than not use encryption, in fact, try it again without encryption and see if the problem still occurs.
Apr 3, 2013 at 3:50 PM
I temporarily got around that Problem with simple try-catching the IOexception with a new .connect and then retrying the command where the connection was closed before. That seems not so elegant to me. I would rather keep the connection than reconnecting every few seconds (of course it hardly depends on the file-situation. I am currently testing my synchronisation with about 2000 files and hundreds of folders.
Coordinator
Apr 3, 2013 at 3:57 PM
If you're not multi-threading your FTP transactions, set EnableThreadSafeDataChannel = false and let me know the results as well. It almost sounds as if the primary control connection is sitting idle (and maybe being disconnected?).
Apr 3, 2013 at 4:01 PM
jptrosclair wrote:
I've seen issues like this with SslStream when running with batch processes. I don't know of a solution other than not use encryption, in fact, try it again without encryption and see if the problem still occurs.
Since I force explicit FTPS from client-side and server-side, disabling the encryption is no solution for me. I have to ensure at least some kind of security for users. Is there a server setting that I could change, to delay the disconnection-timing when large amounts of small files are being transfered or hundreds of folders are being created or any other spammy-command are coming to the server?
Apr 4, 2013 at 8:30 AM
EnableThreadSafeDataChannel = false does not seem to help me. About the same timing. I implemented a counter to see at which count of folders created my FileZilla server kills the connection. I did 5 runs with slightly different folder situations. I deleted the superfolder of the folder where it crashes, to create a slightly new scenario:
  1. 78 2. 78 3. 78 ...
Then I created a completely new file + folder scenario with exact 77 folders. The folder creation works. The sync works until the Upload of the 138th File (of 233).
Then I deleted files until I got 135. Full synchronisation.
After that I raised the folder count to 81. It crashes on folder creation at the 78th folder.
Then I decreased again to 77 folders and raised the file count to 139. Folder creation works, Uploading crashed at the 138th file.

So now I know the two magic numbers 77 folders and 137 files. Since I am connecting to my localhost FileZilla Server, other servers that are outside of the machine or my internal network might not disconnect so fast, because of the lower connection speed.
But how can I increase the amount of folders and files I can push through? The limit I face now is not high enough for my aspirations.
Apr 4, 2013 at 10:05 AM
Edited Apr 4, 2013 at 10:05 AM
I am a bit confused. I just put a using ftpclient around the loop at the folder-creation area and now it works like a swiss clock. Even with 470 folders and > 2000 files in my current scenario. I am not sure how i fixed the Uploading-Problem, perhaps it also had something to do with the folder creation that is managed before the uploading takes place.
On the other hand I deleted the using ftpclient in my Upload-Method. Nevertheless I am happy that it works at this very moment. I think I am not quite done with this problem.