So, I do data processing at work every so often that requires me to copy a single file from multiple data directories into a single target folder prior to the analysis. For only a couple data sets, this isn't a big deal, but every so often I have a good number and it gets tedious.

I'm interested in writing a program that will automate this process by allowing me to input a list of directories (preferably selected via a typical explorer-style window) and a target directory, and it will automatically copy the files over. I'm sure something like this would be easy enough, but I'm not sure what programming language would be best for the job.

Ideally it would have a simple GUI and work across different versions of Windows, but it at least has to function in XP. Any tips on where to start looking? I don't know if it would be easy to write something that is a self-contained .exe file. I only have a limited background in computational programming (Fortran/Matlab), so this is a bit new for me... but I'm looking forward to learning a bit while doing this. Thanks for the advice.

4/13/2011 11:18:07 PM

is it always the same source directories?

if so, you can just write a batch script to do this VERY easily.

4/13/2011 11:26:14 PM

can batch files read XML yet?

Either way you could put the list into a text file broken by 1 directory per line and a batch file would eat this for breakfast. You could probably program it on your lunch break. Or hang around and someone bored at work might find time.


@echo off
for /F "delims=" %%i in (directorylist.txt) do (
echo %%i
copy "%%i" c:\myspecialFolder
)

If it's on a network you can add in ping 127.0.0.1 -n 1 -w 500 > null to wait 1/2 second between requests

[Edited on April 14, 2011 at 12:07 AM. Reason : credit: http://www.hardwareanalysis.com/content/topic/67012/]

4/13/2011 11:54:21 PM

Powershell EZ Mode FTW

Put the names of directories to copy into a file. This script will go line by line and copy the full contents of each directory into your specified directory. I'm on a linux box at home so I can't really test this out until I get to work.

> get-content list_of_directories.txt |% {cp "$($_)\*" "C:\Target_Directory_Name" -recurse}


You could do the same thing pretty easily with C# .NET using the File & Directory class static methods in < 100 lines (not counting the GUI code)

[Edited on April 14, 2011 at 7:59 AM. Reason : correction]

4/14/2011 7:58:26 AM

The data directory names change every time depending on the testing parameters and sample name. Though the folder has different names, the names of the subdirectory path is always the same, and its always a single file type I need to copy over.

It would be something like this: different_data_file_name\Samename1\Samename2\*.m

There's only one .m file in that folder, so it would easy to select only that one. A script seems easy enough, but I was hoping it would be something someone else could come in an use with relative ease as well, thus the idea of a small self-contained .exe with a simple GUI or prompt.

Also, this is a really dumb question, but if I go the script route, what's the easiest way to copy a bunch of folder names at once into a text file? You can't just highlight, copy, and paste (or at least I couldn't do it just now), and doing it for each one from the path line in explorer would defeat the purpose, really.

4/14/2011 8:08:59 AM

I really like this free utility, DSynchronize:

http://dsynchronize.en.softonic.com/

4/14/2011 8:39:04 AM

if the root folder names change, it gets a lot more difficult from a script perspective.

You could write a pretty simple vb app to do this, using a selectable folder list that passes those folders to a function that handles the rest. Go get the free vb express version of visual studio, you can probably learn vb AND write the app in a day

4/14/2011 11:44:09 AM

robocopy batch file + txt file(or maybe 2) for source and destination directories?

4/14/2011 11:47:43 AM

^^ I've seen my coworker use a bit of VB to program report template creators and other excel quantitation stuff. It looked easy enough, I just need to familiarize myself with the basics, and I guess it would be good to have him as a resource if I get stuff... plus then maybe I could add some more to his projects.

4/14/2011 11:24:10 PM

Are you using MATLAB?

Get the uipcikfiles function from the fileexchange:
http://www.mathworks.com/matlabcentral/fileexchange/10867-uipickfiles-uigetfile-on-steroids

If so, you can do this natively inside MATLAB, GUI and all, using the UIGETDIR function.

Example:

%move data script
%data types and order of array dimensions may be all wrong

inputFolders = uigetdir('/my/data/path/','Select Input Folders'); %I don't know if this allows multiple folder selection
outputFolder = uigetdir('/my/destination/path/','Select Destination Folder');

for i=size(inputFolders,1)
copyfile([inputFolders(1,*) '*.m'], outputFolder);
end

4/14/2011 11:54:00 PM

This is a trivial python script. Matlab should not be used for scripting.

4/15/2011 7:51:50 AM

if you need to get a list of folders in to a text file
from a dos prompt

dir /A /b > dirlist.txt


that gives you directories in current directory. You can search/replace to add the beginning path to each line and i'm pretty sure there is a switch to include that in the dir listing, but i can't remember it right now.

4/15/2011 9:46:44 AM

^It's called the NTVDM on Windows 2000 and later, which were derivatives of the Windows NT kernel and not based on MS-DOS.

4/15/2011 10:14:11 AM

You could use the FileSystemWatcher to monitor the root directory for newly generated folders and process as needed.
http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx

Assuming: different_data_file_name\Samename1\Samename2\*.m
I'd guess those are all within some other folder for a longer path, say "c:\data\different_data_file_name\Samename1\Samename2\*.m"
You place a FileSystemWatcher on "c:\data" then whenever a new folder is created or one is altered, you'd verify the folder has what you are looking for ("\Samename1\Samename2\*.m"), then process accordingly (in this case it seems you would simply copy to your other "processing" directory)

It really is hard to answer without fully understanding the the workflow but we regularly use this type of utility.

4/15/2011 4:09:05 PM