Encapsulating

crackmes.de – monkey keygen #1

I’ve just created a keygen for the “monkey keygen #1” challenge on crackmes.de, it was fun and far from “boring crap” which it has been rated as on the site. If this challenge is considered by some to be easy to keygen then consider me humbled, this was a difficult and rewarding challenge for me. This post is going to be long and technical, if you don’t know the difference between eax and al then I suggest you stop reading here :)

Being fairly new to the RE game I decided to search for a string which indicated a valid key had been entered into the program. I managed to find this string at 0x46666C, contents being ‘Valid Key’. That’s cool, let’s see all references to this string:

UMessageBoxValidKey is a custom name I assigned to the subroutine when I had established this was responsible for displaying the message box. Okay, let’s see what’s going on in the UMessageBoxValidKey subroutine:

CODE:00466580 ; ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦ S U B R O U T I N E ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦
CODE:00466580
CODE:00466580 ; Attributes: bp-based frame
CODE:00466580
CODE:00466580 UMessageBoxValidKey proc near           ; DATA XREF: CODE:00466521o
CODE:00466580
CODE:00466580 var_18          = dword ptr -18h
CODE:00466580 var_14          = dword ptr -14h
CODE:00466580 var_10          = dword ptr -10h
CODE:00466580 var_C           = dword ptr -0Ch
CODE:00466580 var_8           = dword ptr -8
CODE:00466580 var_4           = dword ptr -4
CODE:00466580
CODE:00466580                 push    ebp
CODE:00466581                 mov     ebp, esp
CODE:00466583                 add     esp, 0FFFFFFE8h
CODE:00466586                 xor     ecx, ecx
CODE:00466588                 mov     [ebp+var_18], ecx
CODE:0046658B                 mov     [ebp+var_14], ecx
CODE:0046658E                 mov     [ebp+var_C], ecx
CODE:00466591                 mov     [ebp+var_10], edx
CODE:00466594                 mov     [ebp+var_4], eax
CODE:00466597                 xor     eax, eax
CODE:00466599                 push    ebp
CODE:0046659A                 push    offset loc_466660
CODE:004665BD                 xor     eax, eax
CODE:004665BF                 push    ebp
CODE:004665C0                 push    offset sub_466636
CODE:004665C5                 push    dword ptr fs:[eax]
CODE:004665C8                 mov     fs:[eax], esp
CODE:004665CB                 lea     eax, [ebp+var_C]
CODE:004665CE                 push    eax
CODE:004665CF                 lea     edx, [ebp+var_14]
CODE:004665D2                 mov     eax, [ebp+var_4]
CODE:004665D5                 mov     eax, [eax+2FCh]
CODE:004665DB                 call    sub_4320E8
CODE:004665E0                 mov     edx, [ebp+var_14]
CODE:004665E3                 mov     cx, 4DE1h
CODE:004665E7                 mov     eax, [ebp+var_8]
CODE:004665EA                 call    sub_466230
CODE:004665EF                 lea     edx, [ebp+var_18]
CODE:004665F2                 mov     eax, [ebp+var_4]
CODE:004665F5                 mov     eax, [eax+300h]
CODE:004665FB                 call    sub_4320E8
CODE:00466600                 mov     eax, [ebp+var_18]
CODE:00466603                 mov     edx, [ebp+var_C]
CODE:00466606                 call    UCheckValidCombo
CODE:0046660B                 jnz     short loc_466620
CODE:0046660D                 push    0               ; uType
CODE:0046660F                 push    offset aValidKey ; "Valid Key"
CODE:00466614                 push    offset aValidKey ; "Valid Key"
CODE:00466619                 push    0               ; hWnd
CODE:0046661B                 call    MessageBoxA_0
CODE:00466620
CODE:00466620 loc_466620:                             ; CODE XREF: UMessageBoxValidKey+8Bj
CODE:00466620                 xor     eax, eax
CODE:00466622                 pop     edx
CODE:00466623                 pop     ecx
CODE:00466624                 pop     ecx
CODE:00466625                 mov     fs:[eax], edx
CODE:00466628                 push    offset loc_46663D
CODE:0046662D
CODE:0046662D loc_46662D:                             ; CODE XREF: CODE:0046663Bj
CODE:0046662D                 mov     eax, [ebp+var_8]
CODE:00466630                 call    sub_4030FC
CODE:00466635                 retn
CODE:00466635 UMessageBoxValidKey endp ; sp = -30h

Okay so if this challenge allowed for patching, all that would be necessary to fool the program into thinking you had entered a correct key would be to NOP out the jnz op at 0x0046660B. Unfortunately for us the task is to create a keygen, so we need to find the code responsible for generating and checking the key. UCheckValidCombo looks like a good place to start.

CODE:004043AC ; ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦ S U B R O U T I N E ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦
CODE:004043AC
CODE:004043AC
CODE:004043AC UCheckValidCombo proc near              ; CODE XREF: sub_414450+6Bp
CODE:004043AC                                         ; sub_418FBC+Ep ...
CODE:004043AC                 push    ebx
CODE:004043AD                 push    esi
CODE:004043AE                 push    edi
CODE:004043AF                 mov     esi, eax
CODE:004043B1                 mov     edi, edx
CODE:004043B3                 cmp     eax, edx
CODE:004043B5                 jz      loc_40444A
CODE:004043BB                 test    esi, esi
CODE:004043BD                 jz      short loc_404427
CODE:004043BF                 test    edi, edi
CODE:004043C1                 jz      short loc_40442E
CODE:004043C3                 mov     eax, [esi-4]
CODE:004043C6                 mov     edx, [edi-4]
CODE:004043C9                 sub     eax, edx
CODE:004043CB                 ja      short loc_4043CF
CODE:004043CD                 add     edx, eax
CODE:004043CF
CODE:004043CF loc_4043CF:                             ; CODE XREF: UCheckValidCombo+1Fj
CODE:004043CF                 push    edx
CODE:004043D0                 shr     edx, 2
CODE:004043D3                 jz      short loc_4043FB
CODE:004043D5
CODE:004043D5 loc_4043D5:                             ; CODE XREF: UCheckValidCombo+45j
CODE:004043D5                 mov     ecx, [esi]
CODE:004043D7                 mov     ebx, [edi]
CODE:004043D9                 cmp     ecx, ebx
CODE:004043DB                 jnz     short loc_404435
CODE:004043DD                 dec     edx
CODE:004043DE                 jz      short loc_4043F5
CODE:004043E0                 mov     ecx, [esi+4]
CODE:004043E3                 mov     ebx, [edi+4]
CODE:004043E6                 cmp     ecx, ebx
CODE:004043E8                 jnz     short loc_404435
CODE:004043EA                 add     esi, 8
CODE:004043ED                 add     edi, 8
CODE:004043F0                 dec     edx
CODE:004043F1                 jnz     short loc_4043D5
CODE:004043F3                 jmp     short loc_4043FB
CODE:004043F5 ; ---------------------------------------------------------------------------
CODE:004043F5
CODE:004043F5 loc_4043F5:                             ; CODE XREF: UCheckValidCombo+32j
CODE:004043F5                 add     esi, 4
CODE:004043F8                 add     edi, 4
CODE:004043FB
CODE:004043FB loc_4043FB:                             ; CODE XREF: UCheckValidCombo+27j
CODE:004043FB                                         ; UCheckValidCombo+47j
CODE:004043FB                 pop     edx
CODE:004043FC                 and     edx, 3
CODE:004043FF                 jz      short loc_404423
CODE:00404401                 mov     ecx, [esi]
CODE:00404403                 mov     ebx, [edi]
CODE:00404405                 cmp     cl, bl
CODE:00404407                 jnz     short loc_40444A
CODE:00404409                 dec     edx
CODE:0040440A                 jz      short loc_404423
CODE:0040440C                 cmp     ch, bh
CODE:0040440E                 jnz     short loc_40444A
CODE:00404410                 dec     edx
CODE:00404411                 jz      short loc_404423
CODE:00404413                 and     ebx, 0FF0000h
CODE:00404419                 and     ecx, 0FF0000h
CODE:0040441F                 cmp     ecx, ebx
CODE:00404421                 jnz     short loc_40444A
CODE:00404423
CODE:00404423 loc_404423:                             ; CODE XREF: UCheckValidCombo+53j
CODE:00404423                                         ; UCheckValidCombo+5Ej ...
CODE:00404423                 add     eax, eax
CODE:00404425                 jmp     short loc_40444A
CODE:00404427 ; ---------------------------------------------------------------------------
CODE:00404427
CODE:00404427 loc_404427:                             ; CODE XREF: UCheckValidCombo+11j
CODE:00404427                 mov     edx, [edi-4]
CODE:0040442A                 sub     eax, edx
CODE:0040442C                 jmp     short loc_40444A
CODE:0040442E ; ---------------------------------------------------------------------------
CODE:0040442E
CODE:0040442E loc_40442E:                             ; CODE XREF: UCheckValidCombo+15j
CODE:0040442E                 mov     eax, [esi-4]
CODE:00404431                 sub     eax, edx
CODE:00404433                 jmp     short loc_40444A
CODE:00404435 ; ---------------------------------------------------------------------------
CODE:00404435
CODE:00404435 loc_404435:                             ; CODE XREF: UCheckValidCombo+2Fj
CODE:00404435                                         ; UCheckValidCombo+3Cj
CODE:00404435                 pop     edx
CODE:00404436                 cmp     cl, bl
CODE:00404438                 jnz     short loc_40444A
CODE:0040443A                 cmp     ch, bh
CODE:0040443C                 jnz     short loc_40444A
CODE:0040443E                 shr     ecx, 10h
CODE:00404441                 shr     ebx, 10h
CODE:00404444                 cmp     cl, bl
CODE:00404446                 jnz     short loc_40444A
CODE:00404448                 cmp     ch, bh
CODE:0040444A
CODE:0040444A loc_40444A:                             ; CODE XREF: UCheckValidCombo+9j
CODE:0040444A                                         ; UCheckValidCombo+5Bj ...
CODE:0040444A                 pop     edi
CODE:0040444B                 pop     esi
CODE:0040444C                 pop     ebx
CODE:0040444D                 retn
CODE:0040444D UCheckValidCombo endp

So, the cmp at 0x004043B3 is interesting because the EDX register appears to contain a string which could be the key. Let’s test this..

Let’s enter the “2C6B91” string as the key with the name “aaa”.

Yay, that’s not terribly good practice, storing the answer in a string like that. It would be more convoluted in ‘real life’ one would hope. Okay, let’s insert a write break so we know which subroutine is responsible for populating this part of memory. So far we can either patch or generate keys using the challenge binary to bypass it, but we really need to create our own binary to prove that we understand how the key is created (something we don’t yet know).

CODE:004028B8 ; ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦ S U B R O U T I N E ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦
CODE:004028B8
CODE:004028B8
CODE:004028B8 sub_4028B8 proc near                    ; CODE XREF: sub_4025F8+81p
CODE:004028B8                                         ; sub_402A08+14p ...
CODE:004028B8 push    esi
CODE:004028B9 push    edi
CODE:004028BA mov     esi, eax
CODE:004028BC mov     edi, edx
CODE:004028BE mov     eax, ecx
CODE:004028C0 cmp     edi, esi
CODE:004028C2 ja      short loc_4028D7
CODE:004028C4 jz      short loc_4028F5
CODE:004028C6 sar     ecx, 2
CODE:004028C9 js      short loc_4028F5
CODE:004028CB rep movsd
CODE:004028CD mov     ecx, eax
CODE:004028CF and     ecx, 3
CODE:004028D2 rep movsb
CODE:004028D4 pop     edi
CODE:004028D5 pop     esi
CODE:004028D6 retn
CODE:004028D7 ; ---------------------------------------------------------------------------
CODE:004028D7
CODE:004028D7 loc_4028D7:                             ; CODE XREF: sub_4028B8+Aj
CODE:004028D7 lea     esi, [ecx+esi-4]
CODE:004028DB lea     edi, [ecx+edi-4]
CODE:004028DF sar     ecx, 2
CODE:004028E2 js      short loc_4028F5
CODE:004028E4 std
CODE:004028E5 rep movsd
CODE:004028E7 mov     ecx, eax
CODE:004028E9 and     ecx, 3
CODE:004028EC add     esi, 3
CODE:004028EF add     edi, 3
CODE:004028F2 rep movsb
CODE:004028F4 cld
CODE:004028F5
CODE:004028F5 loc_4028F5:                             ; CODE XREF: sub_4028B8+Cj
CODE:004028F5                                         ; sub_4028B8+11j ...
CODE:004028F5 pop     edi
CODE:004028F6 pop     esi
CODE:004028F7 retn
CODE:004028F7 sub_4028B8 endp

The rep movsb instruction at 0x004028F2 copies strings from [ESI] to [EDI], so we need to insert another write break at the ESI address specified (0x0012F5B9).

I needed to repeat this process a couple of times to find the subroutine which was responsible for checking the key, but eventually I determined it was sub_466230. If you look in the screenshot it’s a couple of subroutines up from the routine responsible for the 0x0012F5B9 write. Here’s sub_466230, the heart of the program.

CODE:00466230 ; ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦ S U B R O U T I N E ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦
CODE:00466230
CODE:00466230 ; Attributes: bp-based frame
CODE:00466230
CODE:00466230 sub_466230      proc near               ; CODE XREF: UMessageBoxValidKey+6Ap
CODE:00466230
CODE:00466230 var_1C          = dword ptr -1Ch
CODE:00466230 var_18          = dword ptr -18h
CODE:00466230 var_12          = word ptr -12h
CODE:00466230 var_10          = dword ptr -10h
CODE:00466230 var_B           = byte ptr -0Bh
CODE:00466230 var_A           = word ptr -0Ah
CODE:00466230 var_8           = dword ptr -8
CODE:00466230 var_4           = dword ptr -4
CODE:00466230 arg_0           = dword ptr  8
CODE:00466230
CODE:00466230                 push    ebp
CODE:00466231                 mov     ebp, esp
CODE:00466233                 add     esp, 0FFFFFFE4h
CODE:00466236                 push    ebx
CODE:00466237                 xor     ebx, ebx
CODE:00466239                 mov     [ebp+var_1C], ebx
CODE:0046623C                 mov     [ebp+var_A], cx
CODE:00466240                 mov     [ebp+var_8], edx
CODE:00466243                 mov     [ebp+var_4], eax
CODE:00466246                 xor     eax, eax
CODE:00466248                 push    ebp
CODE:00466249                 push    offset sub_4662E9
CODE:0046624E                 push    dword ptr fs:[eax]
CODE:00466251                 mov     fs:[eax], esp
CODE:00466254                 mov     ax, [ebp+var_A]
CODE:00466258                 mov     [ebp+var_12], ax
CODE:0046625C                 mov     eax, [ebp+arg_0]
CODE:0046625F                 call    sub_403FA0
CODE:00466264                 mov     eax, [ebp+var_8]
CODE:00466267                 call    sub_404260
CODE:0046626C                 test    eax, eax
CODE:0046626E                 jle     short loc_4662D3
CODE:00466270                 mov     [ebp+var_18], eax
CODE:00466273                 mov     [ebp+var_10], 1
CODE:0046627A
CODE:0046627A loc_46627A:                             ; CODE XREF: sub_466230+A1j
CODE:0046627A                 mov     eax, [ebp+var_8]
CODE:0046627D                 mov     edx, [ebp+var_10]
CODE:00466280                 mov     al, [eax+edx-1]
CODE:00466284                 movzx   edx, [ebp+var_12]
CODE:00466288                 shr     edx, 8
CODE:0046628B                 xor     al, dl
CODE:0046628D                 mov     [ebp+var_B], al
CODE:00466290                 xor     eax, eax
CODE:00466292                 mov     al, [ebp+var_B]
CODE:00466295                 add     ax, [ebp+var_12]
CODE:00466299                 mov     edx, [ebp+var_4]
CODE:0046629C                 imul    word ptr [edx+4]
CODE:004662A0                 mov     edx, [ebp+var_4]
CODE:004662A3                 add     ax, [edx+6]
CODE:004662A7                 mov     [ebp+var_12], ax
CODE:004662AB                 lea     ecx, [ebp+var_1C]
CODE:004662AE                 xor     eax, eax
CODE:004662B0                 mov     al, [ebp+var_B]
CODE:004662B3                 mov     edx, 2
CODE:004662B8                 call    USubKeyCalcNO
CODE:004662BD                 mov     edx, [ebp+var_1C]
CODE:004662C0                 mov     eax, [ebp+arg_0]
CODE:004662C3                 call    sub_404268
CODE:004662C8                 mov     eax, [ebp+arg_0]
CODE:004662CB                 inc     [ebp+var_10]
CODE:004662CE                 dec     [ebp+var_18]
CODE:004662D1                 jnz     short loc_46627A
CODE:004662D3
CODE:004662D3 loc_4662D3:                             ; CODE XREF: sub_466230+3Ej
CODE:004662D3                 xor     eax, eax
CODE:004662D5                 pop     edx
CODE:004662D6                 pop     ecx
CODE:004662D7                 pop     ecx
CODE:004662D8                 mov     fs:[eax], edx
CODE:004662DB                 push    offset loc_4662F0
CODE:004662E0
CODE:004662E0 loc_4662E0:                             ; CODE XREF: CODE:004662EEj
CODE:004662E0                 lea     eax, [ebp+var_1C]
CODE:004662E3                 call    sub_403FA0
CODE:004662E8                 retn
CODE:004662E8 sub_466230      endp ; sp = -28h

Okay, this key algorithm is quite involved:

1. al = <nth character in name>
2. edx = 0x4DE1 (or eax if 2nd or more character pass)
3. dl = dl >> 8 (0x4D in this case)
4. al = al ^ 0x4D (0x2C in this case)
5. ax = ax + 0x4DE1 (or eax as it was at step 2 if 2nd or more character pass) (0x4ED0 in this case)
6. ax = ax * 0xCE6D = (0xB189)
6. ax = ax + 0x58BF (0x0A48 (which will be edx in the next round))

Should be accurate (might not be). If that dosen’t make sense, just have a look at the C code (which does work, for me at least).

At first, this code didn’t work at all because the bit shifting I was doing was only 2 bits (as indicated by the commented out debug printf). Once I had fixed that it was necessary to uppercase the hex output and ensure that in case the value was less than 16, a 0 is prepended to the string.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s