Why I need to use pydictor ?

Start:

git clone --depth=1 --branch=master https://www.github.com/landgrey/pydictor.git
cd pydictor/
chmod 755 pydictor.py
python pydictor.py

Overview:

Quick to use:

types of generate wordlist（14 types）and descriptions

wordlist type	number	description
base	1	basic wordlist
char	2	custom character wordlist
chunk	3	permutation and combination wordlist
conf	4	based on configuration file wordlist
sedb	5	social engineering wordlist
idcard	6	id card last 6/8 char wordlist
extend	7	extend wordlist based on rules
scratch	8	wordlist based on web pages keywords
passcraper	9	wordlist against to web admin and users
handler	10	handle the input file generate wordlist
uniqifer	11	unique the input file and generate wordlist
counter	12	word frequency count wordlist
combiner	13	combine the input file generate wordlist
uniqbiner	14	combine and unique the input file generate wordlist

function and scope of support wordlist number

function	support number	description
len	1 2 3 4 5 6 7 9 10 11 12 14	lenght scope
head	1 2 3 4 5 6 7 9 10 11 12 14	add items prefix
tail	1 2 3 4 5 6 7 9 10 11 12 14	add items suffix
encode	1 2 3 4 5 6 7 9 10 11 12 14	encode the items
occur	3 4 5 7 9 10 11 12 14	filter by occur times of letter、digital、special chars
types	3 4 5 7 9 10 11 12 14	filter by types of letter、digital、special chars
regex	3 4 5 7 9 10 11 12 14	filter by regex
level	5 7 9	set the wordlist level
leet	5 7 9	1337 mode

usage examples

1: generate the basic wordlsit based on digital lenght of 4

python pydictor.py -base d --len 4 4 --output D:\exits\or\not\dict.txt

2: encode the wordlist

python pydictor.py -base c --len 1 3 --encode b64

3: use d(digital) L(lowercase letter) c(capital letter) generating wordlist

python pydictor.py -base dLc -o /awesome/pwd

4: use customized characters generating wordlist

python pydictor.py -char "abc123._@ " --len 1 3 --tail @site

5: generate permutation and combination wordlist

python pydictor.py -chunk abc ABC 666 . _ @ "'" --head a --tail 123 --encode md5

6. extend wordlist based on rules

1. extend function mainly directed against web application administrator to generate password
2. You can put your own weak password wordlist in wordlist/Web,extend function will auto unique them,new wordlist will contains them
3. You can modify funcfg/extend.conf,set prefix, suffix, prefix + suffix and middle word when extended
4. extend function support leet mode,pick by level and pick by lenght function，you can learn more in the following

write the following information to '/names.txt'

liwell
shelly
bianji
webzhang

run command

python pydictor.py -extend /names.txt --leet 0 1 2 11 21 --level 1 --len 4 16 --occur "<=10" ">0" "<=2" -o /possbile/wordlist.lst

7: id card last 6/8 char wordlist

pydictor.py -plug pid6 --types ">=0" ">=4" ">=0" --encode b64

note: default sex ='all', it decided by lib/data/data.py default_sex, and 'm' is Male, 'f' is Female

8: using passcraper plugin crawl website generating password wordlist based on plain text found and extend rules

1. the rules of passcraper plug and extend function are the same
2. passcraper plug will generate two wordlist，preffix with SCRATCH is raw wordlist by website plain text， and if you feel that there are a lot of unrelated words in the SCRATCH wordlist, you can remove them, and then use the extend function to specify the new file to generate dictionary again.
3. you can modify the funcfg/passcraper_blacklist.conf file，add or delete useless words that need to be filtered out， and also can modify lib/data/data.py file passcraper_filter argument，change the filter regular expressions
4. with same extend function，you can put your weak password in /wordlist/Web，new wordlist will contains them

python pydictor.py -plug passcraper    using default file scraper.sites as multi-input file
python pydictor.py -plug passcraper http://www.example.com

9. using configuration file build dictionary

this function contains all of "-base" and "-char" capacities，and more precise control

python pydictor.py --conf                           using default file funcfg/build.conf build the dictionary
python pydictor.py --conf /my/other/awesome.conf    using /my/other/awesome.conf build the dictionary

note: parsing rules details as following，besides referred to build.conf file

configuration parsing rules details:

 1. the basic unit of parsing is called an parsing element, an parsing element includes five elements, namely: head, character set, length range, encoding, tail, which can be omitted both head and tail;
A standard parsing element:head[characters]{minlength,maxlength}<encode-type>tail，a example parsing element：a[0-9]{4,6}<none>_
Its meaning build a dictionary that  prefix is "a" , character set is 0—9, don't encode,length range is 4—6 and  suffix is "_"
 2. current is support parsing one line
 3. one line can contains 10 parsing elements
such as:[4-6,a-c,A,C,admin]{3,3}<none>_[a,s,d,f]{2,2}<none>[789,!@#]{1,2}<none>,it contains three parsing elements
 4. if annotator "#" in first place, program won't parse this line
 5. conf function can build more precise dictionary up to single char

about character sets：
       You can add the "-" in the middle of character sets beginning and ending to  join them
       and can also use "," to separate multiple character sets, or a single character, or a single string, as an element of the character set;

supported encoding:
      none    don't encode
      b64     base64
      md5     md5 digest algorithm output 32 char
      md516   md5 digest algorithm output 16 char
      sha1    sha1 digest algorithm
      url     urlencode
      sha256  sha256 digest algorithm
      sha512  sha512 digest algorithm

10. handle wordlist's tools

filter tool handler
specify the input file, and output the handled file

python pydictor.py -tool handler /wordlist/raw.txt --len 6 16 --occur "" "=6" "<0" --encode b64 -o /wordlist/ok.txt

safe delete tool shredder

python pydictor.py -tool shredder   delete the currently specified output path(default:results) files and all its dictionary files
python pydictor.py -tool shredder base   delete the files of it's prefix is "BASE" in currently specified output path

prefix(case insensitive) range in 14 items: base,char,chunk,conf,sedb,idcard,extend,handler,uniqifer,counter,combiner,uniqbiner,scratch,passcraper
besides，you can safe shred files or whole directory as following:

python pydictor.py -tool shredder /data/mess
python pydictor.py -tool shredder D:\mess\1.zip

for improving the security delete speed, the default uses 1 times to erase and rewrite，you can modify lib/data/data.py file's file_rewrite_count and dir_rewrite_count value

remove duplicates tool uniqifer

python pydictor.py -tool uniqifer /tmp/my.dic

word frequency statistics tool counter

python pydictor.py -tool counter vs /tmp/mess.txt 100  select 100 words in /tmp/mess.txt file that appear in the most times and output to the terminal and saved to file

note: default choose 100 items to print or save；default separator is:"\n",you can modify counter_split value in lib/data/data.py file

merge dictionary tool combiner

python pydictor.py -tool combiner /my/messdir

remove duplicates after merging tool uniqbiner

python pydictor.py -tool uniqbiner /my/messdir

11: wordlist filter

filter by level function

1. this function is currently only support extend function, passcraper plug, Social Engineering Dictionary Builder
2. default level is 3, the lower level, the lower possibility, the more items
3. modify funcfg/extend.conf file，customized your awesome level rules

python pydictor.py -extend bob adam sarah --level 5

use leet mode

1. this function is currently only support extend, passcraper, Social Engineering Dictionary Builder
2. all default unable to use leet mode, when enable, you can use multiple code at one time
3. SEDB can enable leet mode and set code in SEDB interface
4. enable leet mode cannot make wordlist decrease，it will increase wordlist on the basis of unable to use the leet mode

default leet table

leet char = replace char

a = 4
b = 6
e = 3
l = 1
i = 1
o = 0
s = 5

code

0   default，replace all
1   left-to-right, replace all the first encountered leet char
2                right-to-left, replace all the first encountered leet char
11-19   left-to-right, replace the first encountered leet char to maximum code-10 chars   
21-29   right-to-left, replace the first encountered leet char to maximum code-20 chars

code effection table

code	old string	new string
0	as a airs trees	45 4 41r5 tr335
1	as a airs trees	4s 4 4irs trees
2	as a airs trees	a5 a air5 tree5
11	as a airs trees	4s a airs trees
12	as a airs trees	4s 4 airs trees
13	as a airs trees	4s 4 4irs trees
14	as a airs trees	4s 4 4irs trees
...	as a airs trees	4s 4 4irs trees
21	as a airs trees	as a airs tree5
22	as a airs trees	as a air5 tree5
23	as a airs trees	a5 a air5 tree5
24	as a airs trees	a5 a air5 tree5
...	as a airs trees	a5 a air5 tree5

besides,you also can:

modify /funcfg/leet_mode.conf, add or delete leet table items;   
modify /lib/lib/data.py, extend_leet、passcraper_leet、sedb_leet arguments, choose some functions whether default use leet mode;
modify /lib/data/data.py,leet_mode_code argument, choose default mode code;

filter by occur times of letter、digital、special chars

--occur [scope of occur letter times] [scope of occur digital times] [scope of occur special chars times]

default occur times

"<=99" "<=99" "<=99"

filter by types of letter、digital、special chars

 --types [scope of letter types] [scope of digital types] [scope of special types]

default types

">=0" ">=0" ">=0"

12. social engineering dictionary

python pydictor.py --sedb

                              _ _      _
              _ __  _   _  __| (_) ___| |_ ___  _ __
             | '_ \| | | |/ _` | |/ __| __/ _ \| '__|
             | |_) | |_| | (_| | | (__| || (_) | |
             | .__/ \__, |\__,_|_|\___|\__\___/|_|
             |_|    |___/                         

                   Social Engineering Dictionary Builder
                                            Build by LandGrey
    ----------------------------[ command ]----------------------------
    [+]help desc             [+]exit/quit            [+]clear/cls
    [+]show option           [+]set option arguments [+]rm option
    [+]len minlen maxlen     [+]head prefix          [+]tail suffix
    [+]encode type           [+]occur L d s          [+]types L d s
    [+]regex string          [+]level code           [+]leet code
    [+]output directory      [+]run

    ----------------------------[ option ]----------------------------
    [+]cname                 [+]ename                [+]sname
    [+]birth                 [+]usedpwd              [+]phone
    [+]uphone                [+]hphone               [+]email
    [+]postcode              [+]nickname             [+]idcard
    [+]jobnum                [+]otherdate            [+]usedchar

pydictor SEDB>>

command:

help                reload interface
help desc           view the meaning for each items
exit or quit        exit the program
clear or cls        clear screen
show                view the current settings
set                 set option value
rm                  remove option value
len                 select the length range
head                add prefix
tail                add suffix
encode              encode items
occur               set occur times of letter、digital、special chars
types               set types of letter、digital、special chars
regex               filter by regex
level               select the extend level value
leet                enable leet mode and choose code
output              set output dictionary or file path
run                 build wordlist

if you have some information about someone

information items	value
chinese name	李伟
pinyin name	liwei
simple name	lw
simple name	Lwei
english name	zwell
birthday	19880916
used password	liwei123456.
used password	liwei@19880916
used password	lw19880916_123
used password	abc123456
phone number	18852006666
used phone number	15500998080
home phone	76500100
company phone	010-61599000
email account	[email protected]
email account	[email protected]
email account	[email protected]
email account	[email protected]
home postcode	663321
now place postcode	962210
common nickname	zlili
id card number	152726198809160571
student id	20051230
job number	100563
father birthday	152726195910042816
mother birthday	15222419621012476X
boy/girl friend brithday	152726198709063846
friend brithday	152726198802083166
pet name	tiger
crazy something	games of thrones
special meaning numbers	176003
special meaning chars	m0n5ter
special meaning chars	ppdog

now, use follwing command:

python pydictor.py --sedb
set cname liwei
set sname lw Lwei
set ename zwell
set birth 19880916
set usedpwd liwei123456. liwei@19880916 lw19880916_123
set phone 18852006666
set uphone 15500998080
set hphone 76500100 61599000 01061599000
set email [email protected]
set email [email protected]
set email [email protected]
set email [email protected]
set postcode 663321 962210
set nickname zlili
set idcard 152726198809160571
set jobnum 20051230 100563
set otherdate 19591004 19621012
set otherdate 19870906 19880208
set usedchar tiger gof gamesthrones 176003 m0n5ter ppdog

view the configuration, and build the wordlist

show
run

if you want more items wordlist, use

level 1

and, you want to filter some impossible password,
set the password lenght

len 1 16

at least one letter and at most three special char,

occur ">0" "" "<=3"

and at most two types of special char in one item,

types "" "" "<=2"

finaly, specify the output path, build wordlist again

output D:\awesome\dict\liwei_pass.txt
run

note:

1. you can modify funcfg/sedb_tricks.conf file，change the word transform prefix, suffix and prefix+suffix rules
2. you can put your own individual weak password wordlist in wordlist/SEDB,
3. SEDB some little rules contains extend function