thesis

GitHub repository containing all necessary information about the bachelor's thesis: "Evaluating LLM based web application penetration testing: How does AI improve efficiency?"

Citation

You can reference my thesis the following way:

@masterthesis{bruesemeister2024pentestllm,
	title        = {Evaluating LLM based web application penetration testing: How does AI improve efficiency?},
	author       = {Patrick Bruesemeister},
	year         = 2024,
	month        = {March},
	school       = {Leipzig University of Applied Sciences},
	type         = {Bachelor's Thesis},
	supervisor   = {Andreas Hartmann and Roy Meissner}
}

The @masterthesis tag has been used in combination with the type set to Bachelor's Thesis based on this answer: https://tex.stackexchange.com/a/392841

What can be found here?

• bench folder: Benchmark results for command generation capabilities of LLMs for penetration testing
• evaluation folder: ChainForge files for the command generation evaluation, if desired to reproduce the experiment / base further research on

Folder: evaluation

• More info on the Chainforge files can be found here

bench information

Used models

NAME                           	ID          	SIZE  	MODIFIED     
codellama:7b                   	8fdf8f752f6e	3.8 GB	5 weeks ago - ollama 	
dolphin-mistral:latest         	ecbf896611f5	4.1 GB	2 weeks ago - ollama 	
llama2:13b                     	d475bf4c50bc	7.4 GB	5 weeks ago - ollama 	
llama2:7b                      	78e26419b446	3.8 GB	5 weeks ago - ollama 	
llama2-uncensored:latest       	44040b922233	3.8 GB	2 weeks ago - ollama 	
mistral:latest                 	61e88e884507	4.1 GB	2 weeks ago - ollama 	
orca-mini:3b                   	2dbd9f439647	2.0 GB	2 weeks ago - ollama 	
wizard-vicuna-uncensored:latest	72fc3c2b99dc	3.8 GB	2 weeks ago - ollama 	
wizardlm-uncensored:latest     	886a369d74fc	7.4 GB	3 weeks ago - ollama
gpt-3.5-turbo - openai
gpt-4-turbo-preview - openai

The full tag information on the models can be found in the models.json You can compare your local models to the initially used ones by running python checkmodals.py --models file.json, whereas file.json contains the response you got from the http://localhost:11434/api/tags endpoint

bench: Further information

More information on how to use bench can be found by opening the README

Contact

Feel free to contact me whenever required, you can find all contact information on my GitHub profile and the website linked in my bio. Contact information for my supervisors can be found here:

Prof. Dr. Andreas Hartmann:

https://fdit.htwk-leipzig.de/fakultaet-dit/personen/professoren/prof-dr-andreas-hartmann

Roy Meissner, M. Sc.:

https://fdit.htwk-leipzig.de/fakultaet-dit/personen

Leipzig University of Applied Sciences:

https://htwk-leipzig.de