Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 64 additions & 55 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,26 @@
# utfuzz

`utfuzz` is a Python fuzzing engine. It supports fuzzing of Python code and generation reproducing code for error and regression suites.
`utfuzz` is a Python fuzzing engine, generating ready-to-use unit tests for both error and regression suites.

### Installation
## Prerequisites

`utfuzz` supports Python versions 3.8-3.11
`utfuzz` supports Python 3.83.11.

You can use one of these ways:
1. Install from package archive:
- download [utfuzz.tar.gz](https://disk.yandex.ru/d/lmnsT9DdaxZyAg)
`utfuzz` depends on Java 17 or newer. If you do not have Java installed, `utfuzz` will install it, or you can do it
manually using these instructions:

[Linux](https://docs.oracle.com/en/java/javase/17/install/installation-jdk-linux-platforms.html) |
[Windows](https://docs.oracle.com/en/java/javase/17/install/installation-jdk-microsoft-windows-platforms.html) |
[macOS](https://docs.oracle.com/en/java/javase/17/install/installation-jdk-macos.html)

## Installation

1. Download [utfuzz.tar.gz](https://disk.yandex.ru/d/lmnsT9DdaxZyAg)
<!-- https://github.com/tamarinvs19/utfuzz/raw/master/utfuzz_build/utfuzz.tar.gz?download=) -->
- run
`
python -m pip install utfuzz.tar.gz
`
2. Run:
```shell
python -m pip install utfuzz.tar.gz
```

<!-- 2. Install from GitHub (if you have [`lfs`](https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-git-large-file-storage)): -->
<!-- ```shell -->
Expand All @@ -26,83 +33,85 @@ You can use one of these ways:
<!-- python -m pip install -e ./ -->
<!-- ``` -->

To run `utfuzz` you can use module `python -m utfuzz` or command `utfuzz` with active python environment.

#### Java installation
## Usage

`utfuzz` depends on Java 17 and newer. If you don't have an installed Java `utfuzz` will try to install it or your can install Java by yourself using these instructions:

[Java installation instruction for Linux](https://docs.oracle.com/en/java/javase/17/install/installation-jdk-linux-platforms.html)

[Java installation instruction for Windows](https://docs.oracle.com/en/java/javase/17/install/installation-jdk-microsoft-windows-platforms.html)
You can run `utfuzz` in one of three modes:
* dialogue mode (default)
* config file mode
* CLI arguments mode

[Java installation instruction for MacOS](https://docs.oracle.com/en/java/javase/17/install/installation-jdk-macos.html)
Upon each test generation, settings are saved to the `utfuzz_config.json` file.

### Using `utfuzz`
### Generate tests in a dialogue mode

To run:
By default, `utfuzz` runs in a dialogue mode. You can use prompts and configure test generation, or stick to default options:
```shell
python -m utfuzz
```
or
```shell
utfuzz
```
The resulting tests appear in the autogenerated `utfuzz_tests` folder by default.

You can use `utfuzz` one of three modes:
* dialog mode (default)
* config file mode
* CLI arguments mode
### Generate tests using CLI options

After each test generation settings will be saved to file `utfuzz_config.json`
Turn off the dialogue mode with `–-skip-dialog` and specify the necessary options, for example:
```shell
python -m utfuzz –-skip-dialog -o /mnt/c/Users/username/uftests
```

Priority:
* if `--use_config_file` default values will be from config file
* if not `--use_config_file` default values will be from cli-arguments
* then if not `--skip-dialog` new values will be from dialog
### Generate tests using a configuration file

Edit the `utfuzz_config.json` file and enable the `--use-config-file` option:
```shell
python -m utfuzz --use-config-file
```

### Run tests

Run the generated tests as the usual `unittest` ones.

## Options

```shell
usage: utfuzz [-h] [--skip-dialog] [--use-config-file] [--generate-only-error-suite]
[-j JAVA] [-t TIMEOUT] [-p PROJECT_DIR] [-o OUTPUT_DIR]
[--sys-paths [SYS_PATHS ...]] [--analyze-targets [ANALYZE_TARGETS ...]]
[--requirements-file REQUIREMENTS_FILE] [--debug]

utfuzz is a Python fuzzing engine. It supports fuzzing of Python code and
generation reproducing code for error and regression suites

options:
-h, --help show this help message and exit
--skip-dialog Do not ask parameters before execution
--use-config-file Use config file in current directory
--skip-dialog do not ask for options interactively
--use-config-file use config file from current directory
--generate-only-error-suite
Generate only error suite
-j JAVA, --java JAVA Path to Java executable file or JAVA_HOME
generate only error suite
-j JAVA, --java JAVA path to Java executable file or JAVA_HOME
-t TIMEOUT, --timeout TIMEOUT
Timeout in seconds for test generation process per one
class or group of top-level functions from one file
timeout in seconds for generating tests
per class or group of top-level functions
from one file
-p PROJECT_DIR, --project-dir PROJECT_DIR
Root directory with your code for testing (will be
used for imports and dependencies resolving)
root directory with code under test
(used for imports and dependency resolving)
-o OUTPUT_DIR, --output-dir OUTPUT_DIR
Directory for generated tests collecting
directory for generated tests
--sys-paths [SYS_PATHS ...]
Additional path to find imports(will be added to
`sys.path`, default = project directory) [optional]
additional path to find imports
(will be added to `sys.path`; default = project directory)
--analyze-targets [ANALYZE_TARGETS ...]
List of files or directories for testing, empty means
<<test all>> [optional]
list of files or directories to test;
empty value field means <<test all>>
--requirements-file REQUIREMENTS_FILE
Path to requirements.txt [optional]
--debug Use debug mode
path to requirements.txt
--debug use debug mode
```

See also main website of UnitTestBot project: utbot.org

### UnitTestBot sources
You can change UnitTestBot source java file by changing `jar`-file `utfuzz/utbot-cli-python.jar` (Note: don't change the file name).

After replacing `jar`-file you should reinstall this module.
## UnitTestBot sources

`utfuzz` has the [UnitTestBot fuzzing engine](https://github.com/UnitTestBot/UTBotJava/tree/main/utbot-cli-python) inside. To change the version of the UnitTestBot source,
replace it with the necessary `utfuzz/utbot-cli-python.jar` file (do not change the file name).
Upon replacing, remember to reinstall this module.

See also main [website](https://utbot.org) of UnitTestBot project and [GitHub repository](github.com/UnitTestBot/UTBotJava).
For more information, see the [UnitTestBot website](utbot.org/python) and the related [GitHub repository](github.com/UnitTestBot/UTBotJava).

36 changes: 18 additions & 18 deletions utfuzz/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ def main():
try:
config = find_config(project_dir)
except EnvironmentException:
my_print("Cannot find config file")
my_print("Cannot find config file.")
return
config_params = load_config(config)
java = config_params.get("java", java)
Expand Down Expand Up @@ -91,34 +91,34 @@ def main():
if java_result != JavaResult.ValidJava:
if not args.skip_dialog:
install = read_with_action(
"utfuzz depends on Java 17, install it? (Y/n) ",
"utfuzz depends on Java 17. Would you like to install? (Y/n) ",
check_yes_no_with_default(True),
)
if install:
my_print("Start Java installation...")
java = java_manager.install_java()
my_print(
f"Installed Java 17 to {java}. You can set it by --java argument at the next time."
f"Installed Java 17 to {java}. To set the path to it, use --java argument next time."
)
else:
return

if java is None:
my_print(
"Some problems with Java! Your can set a correct path to Java 17 using argument --java. See "
"installation instruction in README.md"
"Some problems with Java! To set a correct path to Java 17, use --java argument. "
"See installation instructions in README.md."
)
return

my_print(f"Selected Java: {java}")
my_print(f"Current Java: {java}")

# Thirdly we use dialog
if not args.skip_dialog:
my_print(
f"Set timeout in seconds per one class or top-level functions in one file (set empty to choose {timeout}s)"
f"Set timeout in seconds: per one class or top-level functions in one file. Leave empty to choose {timeout} s."
)
timeout = read_with_action(
f"Timeout in seconds (default = {timeout}s): ",
f"Timeout in seconds (default = {timeout} s): ",
check_int_with_default(timeout),
)

Expand All @@ -128,16 +128,16 @@ def main():
)

my_print(
f"Specify files and directories to analyze, print one file/directory in row, empty input "
f"marks the end (without clarification all files "
f"{'from project directory ' if analyze_targets else 'from configuration '} will be analyzed):"
f"Specify files and directories to analyze: print one file/directory in a row; empty input "
f"marks the end (by default, all files "
f"{'from the project directory ' if analyze_targets else 'from configuration '} will be analyzed):"
)
while target := my_read(" * "):
file_path = pathlib.Path(target)
if not file_path.is_absolute():
file_path = project_dir / file_path
if not file_path.exists():
my_print(" ^-- this file doesn't exists")
my_print(" ^-- this file does not exist")
if file_path.is_file():
analyze_targets.append(file_path)
elif file_path.is_dir():
Expand All @@ -151,7 +151,7 @@ def main():
)

generate_only_error_suite = read_with_action(
f'Do you want to generate only error suite? ({"Y/n" if generate_only_error_suite else "y/N"}) ',
f'Do you want to generate only an error suite? ({"Y/n" if generate_only_error_suite else "y/N"}) ',
check_yes_no_with_default(generate_only_error_suite),
)

Expand All @@ -160,9 +160,9 @@ def main():

python_manager = PythonRequirementsManager(project_dir)
if not python_manager.check_python():
my_print("Please use Python 3.8 or newer")
my_print("Please use Python 3.8 or newer.")
return
my_print("Installing python dependencies...")
my_print("Installing Python dependencies...")
python_manager.python_requirements_install()
try:
if requirements_file is None:
Expand All @@ -172,11 +172,11 @@ def main():
except NotFoundRequirementsTxt:
my_print(
"Cannot find requirements.txt file. "
"If your project has python dependencies please write it to requirements.txt"
"If your project has Python dependencies, please specify them in requirements.txt."
)
except MultipleRequirementsTxt:
my_print(
"Too many requirements.txt files found! Please use --requirements_file argument to set right"
"Too many requirements.txt files found! Please use --requirements-file argument to set the right one."
)
return

Expand Down Expand Up @@ -205,7 +205,7 @@ def main():
requirements_file,
)

my_print(f"Found {len(analyze_targets)} python files to analyze")
my_print(f"Found {len(analyze_targets)} Python files to analyze.")
for f in tqdm.tqdm(analyze_targets, desc="Progress"):
test_file_name = f'test_{"_".join(f.relative_to(project_dir).parts)}'
generate_tests(
Expand Down
40 changes: 23 additions & 17 deletions utfuzz/parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,67 +4,73 @@
def parse() -> argparse.Namespace:
parser = argparse.ArgumentParser(
prog="utfuzz",
description="utfuzz is a Python fuzzing engine. It supports fuzzing of Python code and generation reproducing "
"code for error and regression suites",
epilog="See also main website of UnitTestBot project: utbot.org",
description="utfuzz is a Python fuzzing engine, generating ready-to-use unit tests "
"for both error and regression suites.",
# epilog="See also main website of UnitTestBot project: utbot.org",
# epilog="See utbot.org/python for more information.",
)
parser.add_argument(
"--skip-dialog",
action="store_true",
help="Do not ask parameters before execution",
help="do not ask for options interactively",
)
parser.add_argument(
"--use-config-file",
action="store_true",
help="Use config file in current directory",
help="use config file from current directory",
)
parser.add_argument(
"--generate-only-error-suite",
action="store_true",
help="Generate only error suite",
help="generate only error suite",
)
parser.add_argument(
"-j", "--java", help="Path to Java executable file or JAVA_HOME", default="java"
"-j", "--java",
help="path to Java executable file or JAVA_HOME",
default="java"
)
parser.add_argument(
"-t",
"--timeout",
type=int,
default=60,
help="Timeout in seconds for test generation process "
"per one class or group of top-level functions "
help="timeout in seconds for generating tests "
"per class or group of top-level functions "
"from one file",
)
parser.add_argument(
"-p",
"--project-dir",
default=".",
help="Root directory with your code for testing (will be "
"used for imports and dependencies resolving)",
help="root directory with code under test "
"(used for imports and dependency resolving)",
)
parser.add_argument(
"-o",
"--output-dir",
default="utfuzz_tests",
help="Directory for generated tests collecting",
help="directory for generated tests",
)

parser.add_argument(
"--sys-paths",
nargs="*",
default=[],
help="Additional path to find imports"
"(will be added to `sys.path`, default = project directory) [optional]",
help="additional path to find imports "
"(will be added to `sys.path`; default = project directory)",
)
parser.add_argument(
"--analyze-targets",
nargs="*",
default=[],
help="List of files or directories for testing, empty means <<test all>> [optional]",
help="list of files or directories to test; empty value field means <<test all>>",
)
parser.add_argument(
"--requirements-file",
help="Path to requirements.txt [optional]",
help="path to requirements.txt",
)
parser.add_argument("--debug", action="store_true", help="Use debug mode")
parser.add_argument(
"--debug",
action="store_true",
help="use debug mode")
return parser.parse_args()
6 changes: 3 additions & 3 deletions utfuzz/user_interface/reader.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ def check_exists_path(value: typing.Any) -> Status[pathlib.Path]:
path = pathlib.Path(value)
if path.exists():
return Ok(path)
return Fail(f'{value} does not exists')
return Fail(f'{value} does not exist')


def check_exists_path_with_default(default: pathlib.Path) -> typing.Callable[[str], Status[pathlib.Path]]:
Expand All @@ -75,9 +75,9 @@ def check_valid_path(value: typing.Any) -> Status[pathlib.Path]:
try:
path = pathlib.Path(value)
except Exception:
return Fail(f'Invalid path {value}')
return Fail(f'Invalid path: {value}')
return Ok(path)
return Fail(f'Invalid path {value}')
return Fail(f'Invalid path: {value}')


def check_valid_path_with_default(default: pathlib.Path) -> typing.Callable[[str], Status[pathlib.Path]]:
Expand Down