Application shimming vs Import Table recovery

In this post I am sharing a case that I investigated recently, during the tests of my application, PE-sieve. It demonstrates how the shims applied by the operating system can disrupt Imports recovery.

Tested features

Recently I had a new release of PE-sieve. One of the added features was a complete Import Table recovery. From now, if you run the PE-sieve with an option /imp 3 it will collect all the addresses that the scanned module imported from the DLLs in the memory, and construct a new Import Table out of them. It is a very useful feature that many PE dumpers have. It helps i.e. to deal with packed applications. Let’s take an example of UPX: it may compress the Import Table of the payload, and load it dynamically during unpacking.

PE-sieve offers also another, “milder” mode of the recovery (/imp 2). In this case PE-sieve bases on the existing import table, and only reconstruct the erased elements. It can be used i.e. in the following case, when the Thunks were overwritten by the functions addresses during the imports loading:

thunks1.png

PE-sieve is able to recognize the exports that are at those addresses, and fills their names back into the table:

fill_back.png

Test cases

I decided to test my application on some ready-made and well-known examples. I selected Anthracene’s unpacking series, available here.

The first sample (1dbfd12ad3ee39930578b949c6899d0a) looks pretty straight-forward. It is a simple application showing a MessageBox, packed with the help of UPX.

Peculiarity

I run this example on one of my Virtual Machines (Windows 8 64 bit) and the imports got recovered flawlessly (video).

Later, I tried to do the same on a different machine, with another set of patches installed. For some reason, I could not reproduce the valid results. Import recovery “magically” stopped working – the received results were incomplete. When I tried to dump the payload with PE-sieve, using the option /imp 2 , all the functions imported from Kernel32.dll got recovered, but the function from User32: MessageBoxA did not.

First I assumed that it must be a bug in PE-sieve, but it turned out that other applications i.e. Scylla have the same problem: they cannot map this address to any of the exported functions.

I investigated the mentioned address under the debugger, and this is what I saw. The call to a MessageBoxA was proxied via apphelp:

The function used from the apphelp was not present in the export table, so it is logical that the applications recovering imports could not recognize it. So, it is not really a bug in the application – but a problem caused by some peculiar way in which this import was loaded.

Explanation

Of course I wanted to understand what is the reason of such behavior. I suspected that there must be some shimming going on, but why did it happen? I checked other applications importing MessageBoxA, but each of them used this function directly from User32.dll, and apphelp.dll was not used as a proxy.

I started thinking that it may be related with the fact that my test case is a very old application.

The OS Version in the PE header is set to 4 (Win 95):

I made an experiment and”upgraded it”, just by changing the version number:

And it worked! After this simple change, the shim was no longer applied. The application used the import directly, and, as a result PE-sieve (and other applications) were able to properly recognize the function.

So, it turned out that the operating system applies this shim automatically for the backward compatibility with old applications.

Now when I think of it, it looks pretty obvious, but it was not so intuitive when I saw it for the first time, that’s why I decided to document this case. So, just a small heads-up: when the import recovery is not going right, first check if shims are not the reason! I hope you enjoyed my small writeup.

 

Posted in Programming, Uncategorized | Tagged , , , , , , | Leave a comment

PE-bear – version 0.3.9 available

[UPDATE] This release introduced some stability issues, fixed in 0.3.9.5

Hello! Several months have passed since I released PE-bear 0.3.8. Since it was my old, abandoned project, I did not plan to start developing it again. Initially, I got convinced to be adding only bugfixes, treating it rather as a legacy app. However, it started doing pretty good for a “dead” project. It got 15K+ new downloads, has been mentioned in some cool presentations, featured on OALabs, and added to FlareVM. It all made me reconsider my decision. Also, I started getting messages from users requesting new features. Finally, I decided to break what I said before, and prepare another release.

The current one (0.3.9) comes with some new features. You can download it from the main site of the project:

https://hshrzd.wordpress.com/pe-bear/

1. Added Rich Header (viewing and editing), with calculated checksum. Preview:

rich_hdr.png

New PE-bear displays all the fields of RichHeader, and allows for their editing. It automatically calculates and verifies the Checksum, so it can help spotting the cases when the Rich Header was forged.

2. Added support for the new fields in Load Config Directory. Preview:

load_config.png

Since PE-bear is a pretty old project, it was not able to parse the full Load Config Directory, but only its basic form, ending on SEHHandlerCount. Now it supports the extensions introduced in Windows 8.1 and Windows 10.

3. In Debug Directory: parse and display RSDSI Table (including PDB path etc):

debug_dir.png

In the old version, Debug Directory was displayed, but without parsing the structure nested inside. Now, one of the most popular types, including PDB path, is also parsed: you can view the project path, and also edit it.

In addition, project underwent some internal refactoring, and I added some other tiny improvements.

I must say I started enjoying working on PE-bear again, and already got several new ideas that I am planning to implement. So, this release is not gonna be the last.

Big thanks to all of you who motivated me to “resurrect” this project. I hope you will enjoy the new version, and the PE-bear’s comeback. As always, I am open for any comments and suggestions.

Posted in PE-bear, Tools | 2 Comments

How to compile a PIN tool using Visual Studio 2017

UPDATE: the described problems in compiling the default PIN projects seems to be fixed in the new PIN release: 3.10.

PIN (of Intel) is a great platform for dynamic binary instrumentation. I use it on daily for tracing and deobfuscating malware, and I often recommend it to others. Unfortunately, figuring out how to set it up is not so straight-forward. If you want to compile the default projects that are distributed in the package, you may get multiple errors.

I never saw the full process of fixing them documented. I struggled with this myself, and from time to time people approach me asking for help. That’s why, I decided to make a walk-through, describing all the steps I did in order to get a tool compiled.

    • Used PIN package:
      • pin-3.7-97619-g0d0c92f4f-msvc-windows (link)
    • Environment:
      • Microsoft Visual Studio Community 2017 (Version 15.6.5)
      • Windows 8.1 64bit

Step 0 – I downloaded the PIN package and unpacked it into C:\pin\C_pin

I will be compiling MyPinTool, that is a part of the PIN Package:

my_pin_tool

Step 1 – I opened the single tool in Visual Studio and tried to compile it.

32bit

I got an error:

error1

So, I searched the pin main directory, and I found where this file is. It was in “C:\pin\extras\xed-ia32\include\xed” (we need to pick a 32 bit version for a 32 bit build).

32_or_64

So, I included that folder:

additional_include.png

[C/C++] -> [General] -> [Additional Include Directories]

Step 2 – I  tried to compile it again and got another error:

safeseh

So, I went to disable SAFESEH. From:

option

[Linker] -> [Advanced] -> [Image Has Safe Exception Handlers]

I switched to:

disable

[Linker] -> [Advanced] -> [Image Has Safe Exception Handlers] -> [No]

Step 3 – Another attempt of compilation, and another set of errors. This time at linking level:

unresolved_externals

I googled for those errors and I found this blog. Following the advice,  I solved it by adding “crtbeginS.obj” to additional dependencies:

dependencies.png

[Linker] -> [Input] -> [Additional Dependencies] -> add: crtbeginS.obj

And finally! It compiled:

result.png

I can only say that it was the nastiest part of PIN, and now it should go much easier. There are various sample projects included in the package, very helpful in learning the functionality.

To make working with it even easier, I made some scripts that are adding PIN along with my favorite tracer to the context menu. Thanks to them, I can start tracing any EXE just by one click. You can find them here.

run_with_pin

Appendix

Posted in Tutorial | 6 Comments

PE-bear – version 0.3.8 available

It has been a long time since I abandoned PE-bear project (version 0.3.7 was released in 2014!). But due to the fact that it still has new downloads, and I keep getting messages from its users, I understood it would be a shame to leave it without any support. A tool is alive as long as someone wants to use it, so, here is an update for PE-bear.

https://hshrzd.wordpress.com/pe-bear/

As I wrote in the release notes, the latest release fixes several bugs . In this post I will elaborate on the most important changes and illustrate them with examples.

  1. Fixed bugs in parsing Delay Load Imports (64bit)

So, this is the old, incorrect version (example: winnet.dll, 64bit)

delayed_imp_old

And in the new, corrected one:

delayed_imp_new2. Fixed bugs in parsing Load Config Directory (64bit)

This is the old, incorrect version:

load_config_old The fields ProcessHeapFlags and ProcessAffinityMask should be flipped, otherwise their sizes are incorrectly identified. It is fixed in the new release:

ld_cfg_new

3. While adding a new section, the selected access rights were applied only if the section was loaded from the file. Also, in some alignments, there was a cave appearing between the previous section and the added one, that needed to be fixed manually in headers, or otherwise the application won’t run. This all is fixed in the current version.

add_section

Section test added by new version:

added_by_new

I fixed also some other, smaller bugs here and there. So if you like PE-bear, it’s time to update. And if you don’t know it yet, feel free to give it a try, because from now onward I am not gonna leave this app without support, and if you find any bug it will be fixed as soon as possible. However, I will do only minimalistic mantainance, so don’t ask me for some super cool new extra features. (Or maybe I get tempted for more… No, I won’t 😉)

 

Posted in PE-bear | 4 Comments

White Rabbit crackme!

UPDATE: We already got the three winners. Good job guys! However, we are waiting for the writeups to select the reward for the best one – so if you are still in between of doing the crackme, don’t give up!

UPDATE2: We got first writeups! All the upcoming ones will be linked under the section “Writeups”. The submission for the contest closes 20th February.

UPDATE3: Contest closed! The winner is @Eleemosynator with his writeup available here. Both writeups were very good and detailed, so we decided that the second one, by @pieceofsummer, also deserved a distinction and a bonus reward. Big thanks to both authors!


This time I would like to introduce a small contest organized by me and Grant Willcox. I wrote a small crackme and he volunteered to sponsor the rewards. The first 3 solutions will be rewarded by books chosen by the winners.

The crackme is a 32bit PE file. It shouldn’t be too difficult, but I didn’t want to make it boring either, so it has few tricks.

rabbit

Disclaimer: I am not an author of the graphics used in the application, such as ASCII arts, icons and others. I don’t claim any rights to them.

Rules:

You need to find the flag in format flag{...} and submit it ASAP to any of us as a DM on twitter (@hasherezade or @tekwizz123). After we announced that the contest is closed, we would like you to make a writeup explaining how did you solved it.

There will be an additional reward for the best writeup – so even if you was not the fastest, you still have a chance to get a book for free.

If you have any questions, you can write them as comments to this post and I will be answering them. I am not giving hints via private messages – I want the contest to be fair for everyone.

At the end I will publish my own writeup with a detailed explanation.

Download:

https://goo.gl/6iG4Ri (password: crackme)

Mind the fact, that the crackme contains some small obfuscation and malware-like tricks, so it may be flagged by some of the AV systems as malicious. False positives are very common when it comes to crackmes – it can’t be helped, sorry! I recommend you to run it on a Virtual Machine.

6iG4Ri.qr

check_mark  Finished? You can rate it!

Writeups:

Posted in CrackMe | Tagged , | 9 Comments

Unpacking a malware with libPeConv (Pykspa case study)

In one of the recent episodes of “Open Analysis Live!” Sergei demonstrated how to statically unpack the Pykspa Malware using a Python script. If you haven’t seen this video yet, I recommend you to watch, it is available here – and the full series is really cool.

The video inspired me to use the same sample and demonstrate an alternative solution, applying my library, libPeConv . The advantage of using libPeConv is that you don’t have to spend time on understanding and rewriting the unpacking algorithm. Instead, you can import the original unpacking function from the original malware. It can speed up the work and be helpful also in the cases when the function of our interest is obfuscated.

Analyzed sample

bd47776c0d1dae57c0c3e5e2832f13870a38d5fd

Static analysis

The static analysis of this malware is already well demonstrated in the mentioned video. I will just recall the important points to which we are going to refer.

Function definition

The function that is responsible for unpacking is available at RVA 0x4520. It has the following prototype:

header

By analyzing how it is applied, we can find out what are the arguments that should be passed:

unpack_blob

The first one is a blob of data (BYTE*), second – size of the blob (DWORD), next comes the name of the file where the output will be written, and last one is some magic char. This is how the function declaration should look:

int __cdecl *unpack_func(BYTE* blob, DWORD blob_size, LPCSTR lpFileName, char magic_val);

Function arguments

The function is applied twice, to decrypt two blobs of data (I call them blob1 and blob2). Important things to note are: the offsets of the blobs, their sizes and the passed magic values.

Decrypting blob1:

blob1

  • Blob1 RVA: 0xC030
  • Blob1 size: 0x11000

By following the code before the function call, we can find that the last argument (the magic char) must have the value ‘r’.

mov_r

Decrypting blob2:

blob2

  • Blob2 RVA: 0x1D038
  • Blob2 size: 0x50000

Again, the magic value is ‘r’:

val_r

Now we have all the data to implement a static unpacker.

Writing a unpacker

Setting up the project

For this part you need to have Visual Studio, CMake and Git installed.

I already prepared a template that you can use to make a libPeConv-based project, so it is enough to fetch it from my Github: https://github.com/hasherezade/libpeconv_project_template

git clone --recursive https://github.com/hasherezade/libpeconv_project_template.git

Now use the CMake to generate a VisualStudio project:

step1

The malware is 32bit, so it is important to generate a project for 32bit build, otherwise we will not be able to import the sample. Example:

example

Click “Finish” then  “Generate” and finally you can open the project in Visual Studio.

Unpacker’s code

Code of the full unpacker is very short:

https://gist.github.com/hasherezade/21f0858ee713b60070e2f33ffef44b5f

Firstly, we load the original malware (by a function from peconv). We need it to be loaded with all the dependencies and ready to be executed. A function that allows to achieve it is load_pe_executable:

BYTE* peconv::load_pe_executable(LPCSTR path_to_pe, size_t &out_size);

This malware sample has no relocation table, so we not only need it loaded, but it must be loaded at it’s original base. This operation may fail on some runs, so we have to keep it in mind.

size_t v_size = 0;
BYTE *malware = peconv::load_pe_executable(mal_path, v_size);
if (!malware) return -1;

Then, using the known offset and the reconstructed declaration of the unpacking function, we are importing it from the loaded malware.

ULONGLONG func_offset = (ULONGLONG)malware + 0x4520;
unpack_func = (int (__cdecl *) (BYTE*, DWORD, LPCSTR, char)) func_offset;

We also use the known offsets of the blobs, and make pointers to the data. After we called the unpacking function with appropriate arguments, our payloads will be dumped to files with the supplied names.

DWORD res1 = unpack_func((BYTE*)((ULONGLONG) malware + blob1_offset), blob1_size, "blob1_unpack.bin", 'r');
std::cout << "Unpacked blob1, res:" << res1 << std::endl;

DWORD res2 = unpack_func((BYTE*)((ULONGLONG) malware + blob2_offset), blob2_size, "blob2_unpack.bin", 'r');
std::cout << "Unpacked blob2, res:" << res2 << std::endl;

At the end we can free the loaded malware:

peconv::free_pe_buffer(malware, v_size);

That’s all, the unpacker is ready. One last thing we can do is preparing a .bat file that will run the unpacker until the malware get loaded (remember the loading base issue caused by the missing relocation table).

Example of the batch script:

@echo off
:try_load
peconv_project.exe malware.bin
IF NOT ERRORLEVEL 0 GOTO try_load
echo.
pause

The full package (except the malware) is available here:
https://drive.google.com/file/d/1ogRJvhEB_rFV5s9wSYVl7MbAEv6xcgyU/view

Finally, let’s see it in action:

Posted in Malware, Programming, Tutorial | Tagged | Leave a comment

Solving a PyInstaller-compiled crackme

I got this crackme from one of my readers, who asked me for the help in understanding how to solve it. As he wrote in the e-mail, it comes “from last year competition by the CheckPoint company”. I promised to make a writeup, so here it is :). I hope it will benefit others also.

The crackme is for the beginners, so don’t expect any fireworks ;). But it was relaxing and fun to solve.

The crackme can be found here (password: crackme), also available at HA: 8ee7382cfdf632c29df5f2d9d3286614

Overview

This is how the application looks:

pycrackme1

When we run in, it asks for a username:

enter_name

And when we give an invalid one, it responds with a text:

“Go away, you are not me”

The first important step in solving the crackme, is noticing how exactly was it made and what tools are to be applied. As the icon hints, it seems to be an application written in Python and converted into EXE. But let’s confirm it by looking inside. The main process runs another instance of itself:

Let’s attach the debugger to the child process and see the loaded modules:

We can find that indeed Python2.7 is loaded to interpret the code (the module is marked red on the picture).

At this moment we can confirm that this EXE is in reality a wrapper for a Python script. There are several applications that allows to achieve it. Depending on which application produced the wrapping, the output has a bit different format and requires a different decompiler.

The popular converters of Python scripts into EXE format, are, i.e. Py2Exe and PyInstaller. This time, PyInstaller was applied.

Tools required

Step 1 – Unwrapping the exe

Unpacking the EXE is easy with the appropriate tool. In this case I used PyInstallerExtractor, written in Python.

python pyinstxtractor.py pycrackme.exe

This is the output:

[*] Processing pycrackme.exe
[*] Pyinstaller version: 2.1+
[*] Python version: 27
[*] Length of package: 2604972 bytes
[*] Found 20 files in CArchive
[*] Beginning extraction...please standby
[+] Possible entry point: pyiboot01_bootstrap
[+] Possible entry point: black_box
[*] Found 196 files in PYZ archive
[*] Successfully extracted pyinstaller archive: pycrackme.exe
You can now use a python decompiler on the pyc files within the extracted directory

The script directly hints, that the next step will be to use a Pyhon decompiler and turn the obtained pyc files into Python scripts.
It also hints about the possible entry point of the application. This information helps us to find where the code of our interest is located.

Step 2 – Decompiling the pyc

The produced output is stored in the directory corresponding to the name of the input executable. We can see there multiple modules extracted, but the interesting one seems to be the file called “black_box”:

extracted_files

The black_box is a pyc file with a magic number removed, so we can just copy this part  from some other pyc file that we found in the extracted set, i.e.

get_hdr.png

Let’s paste it at the beginning of the black_box:

paste_hdr

After this step we are ready to save it as black_box.pyc and decompile:

decompiler

And here is the result:
https://gist.github.com/hasherezade/5c91433bc2461f59657921004c505e3e#file-black_box-py

The file got decompiled properly, and at this point we can rename it to py and run like any other Python script. We can see the same prompt as it was in the EXE:

enter_py

Looking at the code we can see that is is mildly obfuscated. The important part is hidden in the variable “c” that is obfuscated by ROT13 and compressed by ZLIB:

c = '789p8594po76n34n0p457s8947p85n0p0q31506093s02nn06654912r30p563o50q367p7q0o3o9qo6q37q6s061r182471246q1qq2o9138q8p3rpssp9rqnn246o638315o9s4oq3580n7o3oso71rr915o8p54n4o2o87s4s6q2pq8661868664pnp0o3853s443n1606907rsqro3oq670np348643s8o65778o5r248r0r7or5350r34p40714601snqps985255so43n278q69s582sq27n92554q919682n941rs05034r241pnqs1nrn40q44p147s73popp45268qqrns2o3os206q6s22ps6p441024r136n9373n32q965ss128p507sqpp78qo63orp2s9017p0379235ps8so96ns2n12169prn9nq7745snq47o71rs2173p39163pp45nn49p844o559p37sq390998267qn5n2712416qp7nq4n591s6867p8n5p9ro1062498678q88n1934s3788q8o7o8rrnr656ps162q72s0423277pp618roopr5q4np71q98p93o645qp6pp093n2sqn546s9171rqs59543rr9ns7n0p575r86p54n5626928320855139r672rrp449o601sn0ro69qpq5p8soppoorq6184590rn071303s6n32s82r857qn1n612q4or4pp8p40orpppn18nqr20279p90r30o12191nq4ns50oq11p8p271883srn3n655sn4177816pr659n3pqs6673so31qo5n985n4rnp0q7s1015q850s71psn99s3npsp01snps6516s4n8p16369r319oq2427628onrp882ooopr019r63q3pq4spnq559po661r2r250969r91rq57q4s6656n5op0n4q681qr3r8sn495555r40nqp27p98p199o5622ppq4q9q022s81p04r067op9151q38o6r65o7r399436r76ro6rsq87poo8557r89opq08s213rrqqsn3sn26q652412ss5r9s6308q612r8471r68po780roo2o22o9661q964824o754750s305sr0rr52oqss6367q7oo32o9rp05770p5ps9813r6324so7575068q4on9r833o1s4nr4ps1p2r6662p6pspo4pr9464or0487p0qs6289475o2o8r5rsnoosn5s751o15537q15343568nors5084sr8npq3o57r4r9s15rn3n0p0p803n35s668q55s00pr8sq6ro715r2sqs70ppqro6qs953nr9p44696n0o55sp9s04o8s9s496p6o23349op863838qrp5p5o1r4rs32297492p6s277r6r6sp303o673sp2or47qnr203qp411s61p308o76104pp5soqp08o9n87s9rs3n5sq016orn986o3o7s5qsp04q865q1o0nn8q1908877ro9pqqo979r460q60rr0210oopssopo32s3r023p9q79r994q32r2555nr8613nor196o3o0272o1sn95s95o698686q0pp0r04p324o2rr0o689s9p483p880592627780s376r1pn8120n98610571po9n7ooo593195r03s2495no223q037361p3qn950op4qq0s4so9q77orq05ss66r7ossqrp1n77rs63qs316nso13o1s19sso07060349qs86s1ss31r8poo9r2nss11530q17ss6688o656525078s784s75q95557nsoo4p54395qrs8no43ns8483sn0os39npn1s7nn38qs6op347n63po2s6p55o33q6rn23993sor0q77009679n44n39q09n0sos00070p6111'
exec base64.b64decode(zlib.decompress(binascii.unhexlify(c.decode('rot13'))))

To understand it better what happens here, we should dump the content after the deobfuscation, rather than executing it. In order to do so, I slightly modified the script. I removed the code responsible for executing the second stage, and substituted it with the function that writes the decompressed result into a file. This is my modified version:
https://gist.github.com/hasherezade/5c91433bc2461f59657921004c505e3e#file-black_box_patched-py

Now, once we execute this script, we get the next stage dumped. And this is how it looks:
https://gist.github.com/hasherezade/5c91433bc2461f59657921004c505e3e#file-decoded-py

from hashlib import sha256
from time import sleep
import socket, sys

PASSWORD = "36949"
HASH = sha256(PASSWORD).hexdigest()
USER = 'Nigel'
CODE = "807290"

IPADDR = '104.25.199.31'
PORT = 587

def login():
    print ""
    username = raw_input("Enter First Name: ")
    if username.rstrip(' \n\t') != USER:
        print "Go away! You are not me..."
        sys.exit()

    print "Hello %s, Good to see you!" % USER
    while True:
        password_guess = raw_input("Enter 5-digit password: ")
        print "[DEBUG]: calculating sha-256 hash"
        print "[DEBUG]: comparing with %s's hash: %s"  % (USER, HASH)
        print "[DEBUG]: performing anti-brute-force delay..."
        sleep(5)
        if sha256(password_guess).hexdigest() == HASH:
            print "Password OK!"
            break
        else:
            print "Wrong password!"

    while True:
        s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)
        s.connect((IPADDR, PORT))
        s.send(str(CODE).decode('hex'))
        s.close()

        print "%s, two-factor authentication is required. A one-time code was sent to your email address" % USER
        code_guess = raw_input("Enter code: ")
        sleep(5)
        if code_guess == CODE:
            print "Success! The code is what you're looking for :)"
            break
        else:
            print "Wrong code!"

login()

The script is not further obfuscated.
Once we read it, it’s pretty straight-forward what to do next. So, the username was Nigel. Then, we have to give his password that is 36949 and finaly his code: 807290. This was my final conversation with the crackme confirming that the code is valid.

python decoded.py

Output:

Enter First Name: Nigel
Hello Nigel, Good to see you!
Enter 5-digit password: 36949
[DEBUG]: calculating sha-256 hash
[DEBUG]: comparing with Nigel's hash: 6912863904dab1ddc332a928bf6df7f365bf1131906f3424aa931c6c85595c34
[DEBUG]: performing anti-brute-force delay...
Password OK!
Nigel, two-factor authentication is required. A one-time code was sent to your email address
Enter code: 807290
Success! The code is what you're looking for :

Exactly the same results we get when we talk with the original EXE:
crackme1
So, the final answer is 807290.

Conclusion

This crackme can be solved very easily if we know the few tricks. The most important was to find what are the proper tools to be applied. Once we got them, we could easily decompile the code and read the answer.

Appendix

Posted in CrackMe, Tutorial | Tagged , | 4 Comments