A little more documentation

[turian]

It seems like scanLinks.py is supposed to be run before scanData.py, correct?

However, the README doesn't reflect this.
Also, the header for scanLinks.py doesn't describe what it does.

What is directScan.py? Should this be used? It drops tables tho.
The header in directScan.py is also incorrect.

[faraday]

There's a part about scanLinks.py under Usage in README:

This creates the pagelinks table and records incoming and outgoing link counts.
python scanLinks.py <hgw.xml file from Wikiprep dump>

directScan.py can be used when you want to test with the exact selection of articles from Gabrilovich et al. in 2005 dump (see selected.txt). You should execute directScan.py instead of scanData.py if you want to test this.

And you mean the line:

USAGE: scanData.py <hgw.xml file from Wikiprep>

from the header in directScan.py, right?

[turian]

Oh, my mistake, I didn't see the USAGE section. I thought the "what this contains" at the top was the usage.

I meant the line that you mentioned "USAGE: scanData.py <hgw.xml file from Wikiprep>" in directScan.py. A handful of other python files say that they are scanData.py, even though they are not.

[faraday]

Usage explanations are enhanced. Scripts try to guide the user when provided with wrong/insufficient arguments.
However, I will edit README in the light of your additions since basic DB setup (creating a "wiki" database with UTF8 charset for instance) is not explained currently.