Crack Me If You Can Contest

Back to: Top Teams

hashcat

Link to original writeup (external, pdf)

Resources

Active Members	26
Handles	alotdv, atom, blandyuk, BlowCane, chancas, dakykilla, dropdead, EvilMog, hashtka, Hydraze, K9, kontrast23, legion, m3g9tr0n, NullMode, philsmd, purehate, rurapenthe, s3inlc, Szul, The_Mechanic, ToXiC, undeath, unix-ninja, Xanadrel, xmisery
Software	List Condense (Hash Management team software), hashcat, oclHashcat, Passcovery Suite (for challenge 3), JtR (for challenge 2 and challenge 4), Tons of small scripts to generate plains matching patterns
Hardware	Roughly 90 GPUs and rougly 60 multi-core CPUs.

[Hardware Used]

Most of us were available all the time. We can say we had around 15-20 members 16H/day working on it. We had tons of GPU, both AMD and some NVidias, but this time with much less problems regarding the driver and overheating since we had prepared carefully. However, in this contest, it was not the hardware power that matters. If you have such slow algorithms then the only way to crack multiples of them is with a 100% straight attack and with a set of password candidates that you really expect to hit.

[The first action to prepare the contest]

...was to select a few members that would come up with ways to generate the plains Korelogic wanted us to submit. The main challenge was to not over complicate the resulting plains. At one point, we actually thought about dropping out of the contest because we thought it was impossible to generate plains that were difficult enough for the other teams to crack, yet still within the rules of the contest.

However, we persevered and came up with 31 different ways to generate plains. Only a third of which we considered "hard", a third was "medium" and the last third was "easy".

The base of almost all of our plains was a selection of the rockyou dictionary, which atom came up with.

Wordlist selection process:

We'll use rockyou, here's why:
- Makes minga happy, using a full known wordlist
- We will select only "rare" words from the wordlist, so that other teams will not quickly realize it's from rockyou
- It's big (14 million entries, makes it hard on slow hashes even if other teams realize it was rockyou)
- Assures correct charset from Korelogic
- There is a count-version (see link below), so we can pick the lowest used words
- Picking lowest used words will steer other teams onto a false track, which our intentional goal is to poison their google searches, as this will lead them to the wrong dictionaries
- Results will have typos in it, so by mangling typo'd plains the chances are very high other teams "think" correctly typed word was used so they can't find the mangling algorithm
```
$ wget \
http://downloads.skullsecurity.org/passwords/rockyou-withcount.txt.bz2				
$ bunzip2 rockyou-withcount.txt.bz2				
$ dos2unix -f rockyou-withcount.txt
```

Sort out 8 bit chars

$ /root/hashcat-utils-1.1/req-exclude.bin 16 < \
	rockyou-withcount.txt | sponge rockyou-withcount.txt

Randomly sort it, because original list is sorted alphabetically in 2nd instance
```
$ sort -R rockyou-withcount.txt | \
	sponge rockyou-withcount.txt
```
Remove words that are "too long". The max allowed length is 12, so we use only words of max length 10
```
$ /root/hashcat-utils-1.1/len.bin 1 19 < rockyou-withcount.txt | \
	sponge rockyou-withcount.txt
```

Select only words from low "occurrence" types used for poisoning

$ grep "      2 " rockyou-withcount.txt >  final				
$ grep "      3 " rockyou-withcount.txt >> final				
$ grep "      4 " rockyou-withcount.txt >> final				
$ grep "      5 " rockyou-withcount.txt >> final				
$ grep "      6 " rockyou-withcount.txt >> final				
$ grep "      7 " rockyou-withcount.txt >> final				

$ mv final rockyou-withcount.txt

Now it's safe to remove the counter

$ cut -b9- < rockyou-withcount.txt | sponge \
	rockyou-withcount.txt

Remove user "mutations". This step is optional, but using only clean words gives us easier readable mutated results. This increases the chances that Korelogic will accept them as they are human readable

$ /root/hashcat-utils-1.1/req-exclude.bin 8 < \
	rockyou-withcount.txt | sponge rockyou-withcount.txt
$ /root/hashcat-utils-1.1/req-exclude.bin 4 < rockyou-withcount.txt | \
	sponge rockyou-withcount.txt
$ /root/hashcat-utils-1.1/req-exclude.bin 2 < rockyou-withcount.txt | \
	sponge rockyou-withcount.txt

Now we ensure the remaining words are not just found in rockyou.txt in case the other teams grep for the words instead of using google
- It's better for us if they have multiple matches.
- To make this possible, we need to create a small tool that checks if the remaining words are in a different common wordlist
- To bring any sense into the next action, it's important that the other common wordlists are bigger than rockyou in case other teams select the wrong one
```
$ twodict.pl
```
Using the "twodict.pl" script, we make sure remaining words are also in other common wordlists as well:
```
$ perl twodict.pl wikipedia-wordlist-sraveau-20090325.txt \
	rockyou-withcount.txt > pre-final
```
At this point, there are ~195k words left, example:

puala kalongan palay

gregw spongecakes whippo

cankles jolibois guding

zailor boskoop

One more round, just for safety

$ perl twodict.pl wikipedia-wordlist-sraveau-20121203.txt pre-final | \
	sponge pre-final

At this point, there are ~91k words left, example:

cankles boskoop paltin

benado hubsch qaadir

garmisch adoptada whate

chemney arester aryeh

neata
- There are some "hard" ones and some that seem to be usernames, etc.
- If we put in usernames it's easier for them to recognize the original list is rockyou, but usernames are mostly in english.
- So we will remove all words that include english substrings, names and so on, etc..
- To get there, some more work is needed:
Little perl script to remove lines that match substrings from a different list:
```
$ http://rlimatches.pl
```
- Running this perl script is extremely inefficient, so it will take ages; have patience!
- The output will be written to "rockyou-withcount.txt.left"
- And that's what it is, the stuff that survived the top10k passwords
- This is used to strongify the selection of the remaining plains

First we remove top10k passwords

$ head -10000 /root/dict/untouched/rockyou.txt > top10k.txt				
$ perl rli-matches.pl top10k.txt				
$ mv pre-final.left final

At this point, there are ~82k words left, example:

cankles boskoop paltin

benado hubsch qaadir

One more round, just for safety

$ mv pre-final.left pre-final				
$ head -1000 /root/dict/untouched/facebook-first.txt > fb1k.txt				
$ perl rli-matches.pl fb1k.txt				
$ mv pre-final.left final

At this point, there are ~78k words left, example:

cankles boskoop benado

hubsch qaadir garmisch

adoptada whate chemney

A quick test shows that the remaining words are in many big common dictionaries:

$ grep ^adoptada$ *				
				
rockyou.txt:adoptada				
wikipedia-wordlist-sraveau-20090325.txt:adoptada				
wikipedia-wordlist-sraveau-20121203.txt:adoptada				
				
$ grep ^chemney$ *				
				
facebook-last.txt:chemney				
rockyou.txt:chemney				
wikipedia-wordlist-sraveau-20090325.txt:chemney				
wikipedia-wordlist-sraveau-20121203.txt:chemney

Split the result into 60 x 1k wordlists for further processing
```
$ split -l 1000 final
```
xaa, xab, etc are the final outfiles.
- Theoretically any of the outfiles are equally good, pick whichever you want.
- Searching them with google should result in crap, not in rockyou.
- If that's the case, we successfully poisoned the list, since we didn't modify anything to the original list, so minga can't complain. :)

All password generating code can be found here: https://hashcat.net/events/CMIYC2014/passgen/

The next step was to provide an overview and a platform to compile everything. I set up the google spreadsheet below mainly to keep track of the length distribution.

[The second main part of our preparations]

...was to bring new members to the team. Our headhunter was blandyuk, who did a great job bringing amazing new talents to our team.

For our collaboration during the contest we used the Hash-Exchange-System LC again, which we had been using at the previous contests and has been greatly improved and maintained by Xanadrel.

Our two main ways of communication this time were an IRC-Channel and another Google spreadsheet.

[In advance of the contest]

...we decided to divide our members into 4 groups and assign them algorithms to work on. This soon turned out to be unproductive since Korelogic decided to group their plains up into 5 "companies" and divide the pool of plains into 13-19 algorithms. A few hours into the contest, we therefore decided to change the assignments of our groups to align more accordingly to Korelogic's companies and challenges. The assignments changed again during the second half of the contest several times. The main purpose of this was to mix up the analysis of patterns that everyone did.

[Contest]

Whenever a contest starts, it takes a few hours to find out what hash-types are used, in which format they are, etc., etc. So that was our first action before we started cracking.

With these screenshots you can see the company specific hash-types identified. Note that not all hash-types can be listed as not all hash-types are supported by our system.

[https://hashcat.net/events/CMIYC2014/pics/result_c1c2.png]

[https://hashcat.net/events/CMIYC2014/pics/result_c3c4.png]

[https://hashcat.net/events/CMIYC2014/pics/result_c5vm.png]

Also note, these are screenshots from after the contest, that is why there are results listed.

Once we finished with Free-For-All (FFA) cracking, we began with pattern analysis. This time we spent nearly all of our time on this part as from experience with previous contests it turned out this is the key to win a contest.

Here's the patterns we noticed during the entire contest

Global pattern:

[https://hashcat.net/events/CMIYC2014/pics/pattern_global.png]

Company pattern:

[https://hashcat.net/events/CMIYC2014/pics/pattern_companies.png]

[WPA]

The WPA's found in the companies 2, 3, 4 and 5 at position 14 were in JtR format. That's a bit of a problem as there exists no converter from JtR format to any other format, we would have been forced to use JtR for cracking those WPA.

I think that was a bit a bad decision from KoreLogic. It would have been OK if they simply generated network .cap files. In that case all WPA crackers would have been able to load it.

What I had to do is to patch JtR a bit. Good thing is that JtR internally uses hashcat c-types for WPA cracking. All I had to do was to store the hccap structure once it was loaded by JtR and write it to a file so that hashcat was able to load it:

This patch to wpapsk.h from JtR did it:

FILE *fd = fopen ("lala.hccap", "ab");
// Debug_hccap();
fwrite (&hccap, sizeof (hccap), 1, fd);
fclose (fd);

After that, I simply ran JtR with the KoreLogic format and all handshakes were stored to lala.hccap, per company. The result are 4 big .hccap files, but it's no problem as hashcat can load multiple handshakes at once. Results can be found here: https://hashcat.net/events/CMIYC2014/hccap/

[Challenge2]

As no hashcat software supports cracking of .docx files, we had to use JtR to crack them. The JtR version used is the one maintained at magnums github tree.

For the pattern, it was obvious they were all mangled version of the basewords:

123456 (?)
password
microsoft

There must be more basewords, but we didn't manage to identify them in time. This algo is so damn slow!

Here's how we organized the mask based cracking:

[https://hashcat.net/events/CMIYC2014/pics/challenge2_masks.png]

[Challenge3]

Not much to mention here as all passwords were simple digits. @KoreLogic: was this a result of a banned plain from another pro-team?

Again, as hashcat does not support cracking of .doc files, we had to use JtR to crack them. The only problem here was that the files were unable to be cracked with JtR for some unknown reason. They simply did not crack even when giving the correct password. Due to the time pressure I didn't try to fix the problem.

I simply switched to Passcovery to crack them. That was a bit of a problem too, as Passcovery does not allow multi-hash cracking for .doc files. So I wrote a simple batch-script and called it 1000 times :)

[Challenge4]

Same as Challenge3, ran some dictionaries against it and found patterns based on financial words such as "billing", "expense", "mastercard", "credit", etc. and then produced wordlists based on those basewords discovered.

[Challenge5]

Some of us tried to load the rars into different rar-cracking tools but since it was rarv5 which we figured out afterwards no tool supported it. We managed to convert the hashes with john and load them into john but failed to crack any of them.

[Challenge6]

Same as Challenge5.

[Challenge7]

Challenge 7 appeared to some of the members of team hashcat as a very interesting Challenge. We quickly found out that this only could be Cisco's new scrypt hash type (the other new one that we did see in the wild is type 8 which according to some cisco documents seems to be using PBKDF2-SHA256).

We tried to crack the Challenge 7 hashes with the scrypt configuration N = 16384, r = 1, p = 1 . The base64 table for the encoding/decoding part that we tried to use is; ./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz .

Unfortunately, we did not crack any hash in a reasonable amount of time. But after we saw this tweet from Korelogic ( https://twitter.com/CrackMeIfYouCan/status/497809778580533248 ) that Challenge 7 was now disabled, we completely moved on to crack other Challenges/hashes and did not try further to understand that hash format, nor did we try to crack the hashes anymore.

[VM exploration]

For the virtual machine given in a .ova file, we already knew the password for LUKS from KoreLogic ( https://contest-2014.korelogic.com/VM_Challenge_notes.txt ), but after that, we also needed the root and/or cmiyc user's password. Of course there could be many ways to get to the /etc/shadow file etc., but we decided to just start an Ubuntu live cd in .iso format that we had handy and mounted the encrypted drive from the live cd. After that, we were able to dump /etc/shadow, modify it to change the root user's sha512crypt hash and look around at other interesting files etc. Afterwards, we rebooted and logged in w/ the modified root password.

From then on we were able to find several hints about passwords e.g. within text files starting from the root folder / . Furthermore, with those hints and some configuration files, we quickly identified some patterns like "wppassword123!@#", "cmiycpassword123!@#" etc. We then went on and were able to decrypt the ecryptfs home folder of the cmiyc user. Unfortunately, we didn't find many hashes there, as we were initially expecting to find under that encrypted cmiyc folder. In the end, we can't complain, as the virtual machine did provide us with several hash lists easily obtained, including mysql 4/5, phpass, descrypt, ldaps/ssha and sha512crypt hashes.

[Management during the contest]

...was mainly to keep in contact with Korelogic, submit our finds, and fix our problems which we had a lot of. First off, there was a problem submitting the doc and docx plains. Although Korelogic initially explained on their FAQ a format that suggested to submit them as "filename plain", this did not work. [KoreLogic note: We had updated the instructions in one place, but not updated the example until hashcat pointed out the omission - thanks to them for pointing that out.] Throughout the first half of the contest, Korelogic's DoS-filter kicked in and kept us from submitting and kept our users from viewing the overall contest statistics.

Another big problem for us was the submission of the LM hashes. Since we had to split them into two halves, we were uncertain why we only had 9000 registered cracks, but almost 12000 founds, while other teams had 10000 to 11000 registered cracked. One friendly guy at Korelogic tried to debug this with us, but the contest ended without a solution. [KoreLogic note: We figured out that team hashcat did not understand that a) we were counting cracked logins, not half-LM's cracked, and b) Street teams had a bunch more LMs than Pro teams. We tried to explain this to hashcat and update the website during the contest, but apparently did not do a good job because tired.]

Another problem was the submission of DES, SHA512, mssql and vbulletin hashes, which Korelogic eventually managed to fix.

Additionally, there were hash types/lists which Korelogic's system did not care about at all, like the vm_mysql, but were later happy to include.

The "scrypt" subject is a little controversial, since we spent many hours trying to crack in addition to all of the hard work that atom spent to figure out the correct settings for how the hashes had been produced. Korelogic later just removed these hashes. The scrypt value of 9999 was too high to allow for dupes, which some street teams were receiving credits for after cracking other hash types. On the other hand, with Korelogic just removing these hashes altogether from the contest was also disappointing, considering all of the energy that our team put into this which was now lost time and energy. We would have rather seen just a reduction of the value.

In contrast to our feeling on scrypt, we were happy to see the removal of the grub2 hashes after our inquiry into Korelogic that its' salt were bogus.

[Thoughts about the contest]

...are pretty good despite the initial doubts about the new precontest methods. The process of generating plains for the other teams to crack, while staying within the rules of Korelogic to keep things simple, was painful. It was very unsettling to not know which of our plains would be rejected by penalty and replaced by all digits, and which ones would make it. Despite all of our initial doubts about the teams creating the plains, it was a very pleasurable experience since the overall plains were structured in a way that we managed to keep finding them, which was not the case in other contests. Also, we were very happy communicating with Korelogic's support throughout the duration of the contest. They took great effort into solving every problem that we presented to them.

[Thanks]

Ty Fox :)