diff --git a/sfi/images/gif_crash_reproduce_demo.gif b/sfi/images/gif_crash_reproduce_demo.gif deleted file mode 100644 index 9d87205..0000000 Binary files a/sfi/images/gif_crash_reproduce_demo.gif and /dev/null differ diff --git a/sfi/images/gif_crash_reproduce_demo_small.gif b/sfi/images/gif_crash_reproduce_demo_small.gif deleted file mode 100644 index 35ba516..0000000 Binary files a/sfi/images/gif_crash_reproduce_demo_small.gif and /dev/null differ diff --git a/sfi/images/gif_fuzz_debug.gif b/sfi/images/gif_fuzz_debug.gif new file mode 100644 index 0000000..2c8b76a Binary files /dev/null and b/sfi/images/gif_fuzz_debug.gif differ diff --git a/sfi/images/gif_fuzz_demo_small.gif b/sfi/images/gif_fuzz_demo_small.gif deleted file mode 100644 index ded1258..0000000 Binary files a/sfi/images/gif_fuzz_demo_small.gif and /dev/null differ diff --git a/sfi/images/gif_fuzz_demo.gif b/sfi/images/gif_fuzz_run.gif similarity index 54% rename from sfi/images/gif_fuzz_demo.gif rename to sfi/images/gif_fuzz_run.gif index 9fdaa55..6a2410e 100644 Binary files a/sfi/images/gif_fuzz_demo.gif and b/sfi/images/gif_fuzz_run.gif differ diff --git a/sfi/images/img_fuzz_pserv_screenshot1.png b/sfi/images/img_fuzz_pserv_screenshot1.png index 74c3dba..d8c8b54 100644 Binary files a/sfi/images/img_fuzz_pserv_screenshot1.png and b/sfi/images/img_fuzz_pserv_screenshot1.png differ diff --git a/sfi/images/img_fuzz_pserv_screenshot2.png b/sfi/images/img_fuzz_pserv_screenshot2.png index cbe206a..c51cdc6 100644 Binary files a/sfi/images/img_fuzz_pserv_screenshot2.png and b/sfi/images/img_fuzz_pserv_screenshot2.png differ diff --git a/sfi/images/img_fuzz_pserv_screenshot3.png b/sfi/images/img_fuzz_pserv_screenshot3.png new file mode 100644 index 0000000..23d6d93 Binary files /dev/null and b/sfi/images/img_fuzz_pserv_screenshot3.png differ diff --git a/sfi/images/img_fuzz_results_screenshot1.png b/sfi/images/img_fuzz_results_screenshot1.png new file mode 100644 index 0000000..441793c Binary files /dev/null and b/sfi/images/img_fuzz_results_screenshot1.png differ diff --git a/sfi/images/img_fuzz_results_screenshot2.png b/sfi/images/img_fuzz_results_screenshot2.png new file mode 100644 index 0000000..3153e13 Binary files /dev/null and b/sfi/images/img_fuzz_results_screenshot2.png differ diff --git a/sfi/images/img_fuzz_results_screenshot3.png b/sfi/images/img_fuzz_results_screenshot3.png new file mode 100644 index 0000000..fca4fd5 Binary files /dev/null and b/sfi/images/img_fuzz_results_screenshot3.png differ diff --git a/sfi/images/img_fuzz_results_screenshot4.png b/sfi/images/img_fuzz_results_screenshot4.png new file mode 100644 index 0000000..7f73bed Binary files /dev/null and b/sfi/images/img_fuzz_results_screenshot4.png differ diff --git a/sfi/images/img_fuzz_utils_screenshot1.png b/sfi/images/img_fuzz_utils_screenshot1.png deleted file mode 100644 index 6758880..0000000 Binary files a/sfi/images/img_fuzz_utils_screenshot1.png and /dev/null differ diff --git a/sfi/images/img_fuzz_utils_screenshot2.png b/sfi/images/img_fuzz_utils_screenshot2.png deleted file mode 100644 index b5ecdff..0000000 Binary files a/sfi/images/img_fuzz_utils_screenshot2.png and /dev/null differ diff --git a/sfi/images/img_fuzz_utils_screenshot3.png b/sfi/images/img_fuzz_utils_screenshot3.png deleted file mode 100644 index 43ddfb0..0000000 Binary files a/sfi/images/img_fuzz_utils_screenshot3.png and /dev/null differ diff --git a/sfi/images/img_fuzz_utils_screenshot4.png b/sfi/images/img_fuzz_utils_screenshot4.png deleted file mode 100644 index 0fd7e3a..0000000 Binary files a/sfi/images/img_fuzz_utils_screenshot4.png and /dev/null differ diff --git a/sfi/sfi_after_fuzzing.md b/sfi/sfi_after_fuzzing.md index 36d55ab..8709dbd 100644 --- a/sfi/sfi_after_fuzzing.md +++ b/sfi/sfi_after_fuzzing.md @@ -1,46 +1,33 @@ -# What do I do after fuzzing? (`fuzz-utils.py`) +# What do I do after fuzzing? -Once you've completed a fuzzing run, you'll most likely have a few output files whose contents caused your server to crash or hang. (If the fuzzer didn't report any, congratulations! Your server must be pretty robust.) Each of these files contains the exact input that was sent to your server that caused the issue. For example, from the example shown in `how_to_fuzz.md`, we can see a few of the crash files in `./fuzz_out/fuzz0/crashes`: +Once you've completed a fuzzing run, you'll most likely have a few output files whose contents caused your server to crash or hang. (If the fuzzer didn't report any, congratulations! Your server must be pretty robust.) Each of these files contains the input that was sent to your server that caused the issue. From the example shown in `how_to_fuzz.md`, we can see a few of the crash files in `./fuzz_out_2021-11-16_09-42-39/fuzz1/crashes`: -![](./images/img_fuzz_utils_screenshot1.png) - -We can open one of these files up. Running `cat ./fuzz_out/fuzz0/crashes/id:000000,sig:11,src:000008+000068,time:193665,op:splice,rep:4` shows us the exact message that caused the server to crash: +![](./images/img_fuzz_results_screenshot1.png) +The `LD_PRELOAD` library ("sockstorm") developed for this purpose uses a special file format to represent several connections' data in a single run. Because of this, sending the file straight to your server won't reproduce the exact behavior found by the fuzzer. + +(If you'd like to see the details of one of these **comux** files, run `~cs3214/bin/comux -s -i PATH_TO_FILE [-v]` on one to show a summary of how many connections are represented in the file, and what data will be sent to the server.) + +Let's look at `./fuzz_out_2021-11-16_09-42-39/fuzz1/crashes/id:000000,sig:11,src:000188+000106,time:86,ss_chunk_havoc`. Based on the file's name (we can see `sig:11`), the fuzzer indicated the server crashed by receiving a SIGSEGV signal (a segmentation fault) when this file's contents were sent to the server. + +## Reproducing a Crash or Hang + +Once the fuzzer has found a bug in your code, the next logical step would be to reproduce it and debug. This is made easy by the scripts generated inside the fuzzer's output directory (`fuzz-rerun.sh` and `fuzz-rerun-gdb.sh`) Let's say we want to try recreating the supposed SIGSEGV the server received when that file's contents were sent to the server. + +We can take the command straight from the fuzzing summary and modify it to point to the file we're interested in: + +```bash +$ ./fuzz_out_2021-11-16_09-42-39/fuzz-rerun.sh ./fuzz_out_2021-11-16_09-42-39/fuzz1/crashes/id:000000,sig:11,src:000188+000106,time:86,ss_chunk_havoc ``` -GET /Opi/login HTTP/1.1 -Host: hornbe/1.1 -Host: hornbeam.rloginJ21564 -Accept-Encoding: identity -Content-Length: -GET /api/log 64 -{"username": "user0 +![](./images/img_fuzz_results_screenshot2.png) + +As promised, a segmentation fault occurred! Now, let's try running it in GDB: + +```bash +$ ./fuzz_out_2021-11-16_09-42-39/fuzz-rerun-gdb.sh ./fuzz_out_2021-11-16_09-42-39/fuzz1/crashes/id:000000,sig:11,src:000188+000106,time:86,ss_chunk_havoc ``` - -Based on the name of the file (we can see `sig:11`), your server received a SIGSEGV signal (a segmentation fault). Using `fuzz-utils.py`, we can take a closer look at the file's contents and use it to recreate the same crash. -## Post-Fuzzing Utilities +![](./images/img_fuzz_results_screenshot3.png) -Running `fuzz-utils.py` will produce a help menu, with a few tools you can use in tandem with these crash files: - -![](./images/img_fuzz_utils_screenshot2.png) - -With the `--tool` switch, you can specify a tool to invoke. They're discussed below. - -## Hexdump Tool - -To get a better idea of what's inside each crash/hang-inducing output file, a hexdump tool is built into `fuzz-utils.py`. The contents of a file are printed, in hexadecimal, when the script is invoked like so: `fuzz-utils.py --tool hex `. - -For example, if we take the crash file shown above and run it through the hexdump tool, we get: - -![](./images/img_fuzz_utils_screenshot3.png) - -## Sending Tool - -To actually reproduce a crash/hang found by AFL++, you'll want to send the exact same input to your server. To do this, launch your server in one terminal, and run this in another: `fuzz-utils.py --tool send `. The script will read STDIN and send it through a socket to your server. To send the contents of a file, simply use I/O redirection: `fuzz-utils.py --tool send < `. - -If we consider the same example as before, we can start our server (something like: `./server -p 13650`), then, on the same machine, run `fuzz-utils.py --tool send 127.0.0.1 13650 < ./fuzz_out/fuzz0/crashes/id:000000,sig:11,src:000008+000068,time:193665,op:splice,rep:4`. When the server tries to parse the input, the segmentation fault will occur. - -![](./images/img_fuzz_utils_screenshot4.png) - -With this, you'll be able to debug: launch your server in GCC in one terminal, send the crash file's contents in another, and go to town. +Again, the SIGSEGV occurred. From here, you can debug in GDB to discover the source of your bug. diff --git a/sfi/sfi_concepts_fuzzing.md b/sfi/sfi_concepts_fuzzing.md index 81b88fe..7e9f5b6 100644 --- a/sfi/sfi_concepts_fuzzing.md +++ b/sfi/sfi_concepts_fuzzing.md @@ -10,11 +10,11 @@ Now, to make sure this fruit slicer works correctly, you decide it's best to tes ![](./images/img_fuzzing_analogy1.png) -At this point, you might think your machine is perfect: it cuts every kind of fruit flawlessly. Enter, fuzzing. What happens if you put something into the fruit slicer that it isn't expecting? Some unexpected inputs might be: water, bread, a paper towel, a rock, or a spoon. These aren't fruits, and some of them aren't even food. Will your machine break? +At this point, you might think your machine is perfect: it cuts every kind of fruit flawlessly. Enter, fuzzing. Take the fruit and surround it by a bunch of rocks, then glue it together and bake it in the oven for twelve hours. We've "mutated" the original input to be something odd and unexpected. Will your machine break? ![](./images/img_fuzzing_analogy2.png) -By fuzzing, we can uncover bugs that might later be revealed by attackers or flawed inputs. Identifying such bugs and fixing them will make our programs more robust and secure. +Now, mutate something else (or the same input a second time), and try it again. Now do it again. Over and over. This is fuzzing: creating unexpected inputs and feeding it to our target repeatedly, in some automated fashion. By fuzzing, we can uncover bugs that might later be revealed by attackers or flawed inputs. Identifying such bugs and fixing them will make our programs more robust and secure. ## Fuzzing HTTP Servers @@ -37,4 +37,4 @@ User-Agent: Mozilla/4.0 Conn---ection: Keep-Alivey ``` -As is common with most fuzzers, the input was "mutated" to produce this odd, incorrect HTTP request message. Will your server see the `GONT` and recognize it as an invalid method? Will it realize `HTTP/1.111` is an invalid version number? If your program handles this badly, and ends up crashing or doing something it shouldn't, then fuzzing was a success. An attacker could take advantage of this vulnerability to bring your server down while it's doing important work! Of course, a proactive programmer could also take this knowledge and patch up the problem to prevent future crashes. \ No newline at end of file +As is common with most fuzzers, the input was "mutated" to produce this odd, incorrect HTTP request message. Will your server see the `GONT` and recognize it as an invalid method? Will it realize `HTTP/1.111` is an invalid version number? If your program handles this badly, and ends up crashing or doing something it shouldn't, then fuzzing was a success. An attacker could take advantage of this vulnerability to bring your server down while it's doing important work. Of course, a proactive programmer could also take this knowledge and patch up the problem to prevent future crashes. diff --git a/sfi/sfi_concepts_sockfuzz.md b/sfi/sfi_concepts_sockfuzz.md deleted file mode 100644 index 4e46b59..0000000 --- a/sfi/sfi_concepts_sockfuzz.md +++ /dev/null @@ -1,19 +0,0 @@ -# Concepts: What is sockfuzz? - -AFL and AFL++ are excellent at what they do, but they have limitations. One such limitation is how AFL feeds input to the target program: it only works with programs that read from STDIN or from a file. In many cases, this is sufficient; lots of C programs take their input from STDIN or a file. - -However, this project is about creating a HTTP server. Servers don't read input through a file or STDIN - they read from network sockets. So, the question becomes: how can we force a HTTP server to read input from STDIN, so we can fuzz it with AFL? Additionally, how can we do this without modifying your source code? - -Sockfuzz is a small C library I developed to solve this problem. It works by "overloading" the `accept` system call and running some extra code to establish an internal connection to your server. Using the special `LD_PRELOAD` environment variable, it can convince your server to use sockfuzz's copy of `accept`, rather than the actual system call. - -![](./images/img_sockfuzz_diagram1.png) - -Once called, sockfuzz only allows one thread to finish the call to `accept`. The others are forced to block on a call to `sem_wait`. The one thread that is allowed through runs code that makes a connection to the server, spawns two threads, and calls the _real_ `accept` system call, returning its value. From your point of view, your server behaves just about the same when preloaded with sockfuzz, apart from using only one its threads and setting up that internal connection. - -A screenshot of sockfuzz's overloaded `accept` function shows what your server's threads will do when they call sockfuzz's version of the function: - -![](./images/img_sockfuzz_code1.png) - -The two threads that get spawned are designated as the "input thread" and the "output thread." The input thread's job is to read STDIN (until EOF is reached) and feed it through the open network socket to the server. Once STDIN is exhausted, it exits. The output thread's job is to receive bytes from the network socket and send them straight to STDOUT. Once the connection is closed, this thread exits. Collectively, these two threads form a system to send the contents of STDIN to your server and dump the server's response to STDOUT. - -![](./images/img_sockfuzz_example1.png) diff --git a/sfi/sfi_concepts_sockstorm.md b/sfi/sfi_concepts_sockstorm.md new file mode 100644 index 0000000..b9b14a6 --- /dev/null +++ b/sfi/sfi_concepts_sockstorm.md @@ -0,0 +1,17 @@ +# Concepts: What is sockfuzz? + +AFL and AFL++ are excellent at what they do, but they have limitations. One such limitation is how AFL feeds input to the target program: it only works with programs that read from STDIN or from a file. In many cases, this is sufficient; lots of C programs take their input from STDIN or a file. + +However, this project is about creating a HTTP server. Servers don't read input through a file or STDIN - they read from network sockets. So, the question becomes: how can we force a HTTP server to read input from STDIN, so we can fuzz it with AFL? Additionally, how can we do this without modifying your source code? + +Sockstorm is a C library I developed to solve this problem. It works by "overloading" the `accept` system call and running some extra code to establish an internal connection to your server. Using the special `LD_PRELOAD` environment variable, it can convince your server to use sockfuzz's copy of `accept`, rather than the actual system call. + +## Connection Multiplexing + +Once called, sockstorm's version of the `accept` system call spawns a controller thread. This controller threads reads input via stdin, expecting a specific file format (dubbed the **comux** file format). These comux files are designed to specify the data to be sent to the target server across multiple connections. The controller thread parses the input file, then spawns individual threads to send "chunks" of data to the target server across specific connections. + +This approach allows for multiple internal client connections to be made to your server, increasing the probability of finding multithreading-related bugs. As a bonus, it requires *zero* modification to your source code. All you have to do is prepend `LD_PRELOAD=/path/to/sockstorm-preload.so` to your command-line invocation of your server, then pipe one of these comux files to your process via stdin. + +## AFL++ Custom Mutator + +The other half of sockstorm is an AFL++ custom mutator. AFL++ does great when fuzzing many programs on its own, but for more complex file formats (such as the **comux** files being used here), a custom mutator can be implemented to ensure the file's structure doesn't get overwritten during fuzzing. Sockstorm's mutator (`sockstorm-mutator.so`) does just that; it maintains the structure of each comux file while also randomly modifying (fuzzing) the connection data to be sent to the target server. diff --git a/sfi/sfi_how_to_fuzz.md b/sfi/sfi_how_to_fuzz.md index 75d72db..577f683 100644 --- a/sfi/sfi_how_to_fuzz.md +++ b/sfi/sfi_how_to_fuzz.md @@ -8,22 +8,22 @@ Fuzzing your server is as simple as typing `fuzz-pserv.py --src-dir ` +1. Run `fuzz-pserv.py --src-dir /path/to/your/pserv/src` 2. Wait for it to finish (might be a while), or hit Ctrl-C once you're satisfied. 3. Look at the summary: if any crashes or hangs are found, use these files to debug. -![](./images/gif_fuzz_demo.gif) +![](./images/gif_fuzz_run.gif) ## Quickstart: Reproducing a Crash/Hang To reproduce a crash found by the fuzzer, do the following: 1. Choose one of the crash files in the output directory. -2. Run your server in one terminal. -3. Open another terminal and run: `fuzz-utils.py --tool send < ` +2. Run the following command: `/path/to/fuzzer_output/fuzz-rerun-gdb.sh < /path/to/crash/file` 4. Investigate! Your server should crash or hang, depending on the file you chose. -![](./images/gif_crash_reproduce_demo.gif) +![](./images/gif_fuzz_debug.gif)