I feel stupid scp

scp -r myserver.com:"/path/with/a/Space\\ In\\ It" ./

Oh...how did I not know this?