Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ __pycache__
site/
scripts/generated_tests/
nohup.out
error.log
*.log
*.pyc
.vscode/settings.json
1 change: 0 additions & 1 deletion docs/520_2.0.0/sdk/solution_kdp2_host_mipi.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ User can edit and debug with Keil MDK for further implementation [keil/MDK docs
# bin_gen.py will concatenate SCPU/NCPU FW, models_520.nef to generate flash_image.bin
# Note that you may need to substitute '/' for '\' in the path
```
```

- **Program the firmware bin image**
Reference: [Jlink programming](../flash_management/flash_management.md#4-program-flash-via-jtagswd-interface)
Expand Down
1 change: 0 additions & 1 deletion docs/520_2.1.0/sdk/solution_kdp2_host_mipi.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ User can edit and debug with Keil MDK for further implementation [keil/MDK docs
# bin_gen.py will concatenate SCPU/NCPU FW, models_520.nef to generate flash_image.bin
# Note that you may need to substitute '/' for '\' in the path
```
```

- **Program the firmware bin image**
Reference: [Jlink programming](../flash_management/flash_management.md#4-program-flash-via-jtagswd-interface)
Expand Down
1 change: 0 additions & 1 deletion docs/520_2.2.0/sdk/solution_kdp2_host_mipi.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ User can edit and debug with Keil MDK for further implementation [keil/MDK docs
# bin_gen.py will concatenate SCPU/NCPU FW, models_520.nef to generate flash_image.bin
# Note that you may need to substitute '/' for '\' in the path
```
```

- **Program the firmware bin image**
Reference: [Jlink programming](../flash_management/flash_management.md#4-program-flash-via-jtagswd-interface)
Expand Down
2 changes: 1 addition & 1 deletion docs/model_training/OpenMMLab/RSN18.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ If mmcv and mmcv-full are both installed, there will be `ModuleNotFoundError`.

# Step 1: Training models on standard datasets

MMPose provides hundreds of existing and existing pose models in [Model Zoo](https://mmpose.readthedocs.io/en/latest/modelzoo.html), and supports several standard datasets like COCO, MPII, FREIHAND, etc. This note demonstrates how to perform common object detection tasks with these existing models and standard datasets, including:
MMPose provides hundreds of existing and existing pose models in [Model Zoo](https://mmpose.readthedocs.io/en/latest/model_zoo.html), and supports several standard datasets like COCO, MPII, FREIHAND, etc. This note demonstrates how to perform common object detection tasks with these existing models and standard datasets, including:

- Use existing models to inference on given images.
- Test existing models on standard datasets.
Expand Down
2 changes: 1 addition & 1 deletion docs/model_training/regression.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ lite_hrnet
```

**For MPII data**, please download from [MPII Human Pose Dataset](http://human-pose.mpi-inf.mpg.de/).
The original annotation files have been converted into json format, please download them from [mpii_annotations](https://openmmlab.oss-cn-hangzhou.aliyuncs.com/mmpose/datasets/mpii_annotations.tar).
The original annotation files have been converted into json format, please download them from [mpii_annotations](https://download.openmmlab.com/mmpose/datasets/mpii_annotations.tar).
Extract them under `$LITE_HRNET/data`, and make them look like this:

```
Expand Down
12 changes: 6 additions & 6 deletions docs/toolchain/appendix/command_line.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,20 +172,20 @@ Here is an example JSON with comments. **Please remove all the comments in the r
// [optional]
// The encryption setting for the batch compiler. Default is not enabled.
"encryption": {
// Whether enable encrytion
// Whether enable encryption
"whether_encryption": false,
// Encrytion mode selection
// Encryption mode selection
// Options: 1, 2
"encryption mode": 1,
// Encrytion key. A hex string. Required in mode 1.
// Encryption key. A hex string. Required in mode 1.
"encryption_key": "0x12345678",
// Encrytion file. An absolute path. Required in mode 1.
// Encryption file. An absolute path. Required in mode 1.
"key_file": "/data1/enc.txt",
// Encrytion key. A hex string. Required in mode 2, optional in mode 1.
// Encryption key. A hex string. Required in mode 2, optional in mode 1.
"encryption_efuse_key": "0x12345678"
},
// [optional]
// Whether seperate buffers for each model output. Default is true.
// Whether separate buffers for each model output. Default is true.
"dedicated_output_buffer": true,
// [optional]
// Whether compress weight for saving space.
Expand Down
4 changes: 2 additions & 2 deletions docs/toolchain/appendix/converters.md
Original file line number Diff line number Diff line change
Expand Up @@ -430,7 +430,7 @@ it is the output of a `Gemm`, there is no need to do any more edition.
the output, then the model is transposed into channel first. We can use the model editor to safely remove the
`Transpose`.

**Step 3:** If the input shape is not availble or invalid, we can use the editor to give it a valid shape.
**Step 3:** If the input shape is not available or invalid, we can use the editor to give it a valid shape.

**Step 4:** The model need to pass `onnx2onnx.py` again after running the editor. See
[section 6](#6-onnx-to-onnx-onnx-optimization).
Expand Down Expand Up @@ -469,7 +469,7 @@ Cutting is not recommended to be done with other operations together.
positional arguments:

in_file input ONNX FILE
out_file ouput ONNX FILE
out_file output ONNX FILE

optional arguments:

Expand Down
12 changes: 6 additions & 6 deletions docs/toolchain/appendix/history.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
* Add Einsum defusion in kneronnxopt.
* Support Cast to int64 in knerex and compiler.
* Support HardSwish, Topk and Split nodes in knerex and compiler.
* Update the regression flow log printing. Print success log seperately from errors to avoid confusing.
* Update the regression flow log printing. Print success log separately from errors to avoid confusing.
* Update IP evaluator for DMA with small length.
* Fix the kneronnxopt bug in `replace_Gather_with_Slice`.
* Fix the knerex bug: node Concat channel mismatch.
Expand All @@ -40,7 +40,7 @@
* Add `const_in_bitwidth_mode` option for quantization. The default is int16. Unless the customer particularly desires to increase the speed, it can be changed to int8
* Update analyzer exception log.
* Update kneronnxopt to set expanding dilated Conv to False by default.
* Update kneronnxopt to diable fusing BatchNormalization into Conv by default.
* Update kneronnxopt to disable fusing BatchNormalization into Conv by default.
* Update compiler for the deep search memory estimation algorithm.
* Update compiler to extend the timeout for deep search.
* Update compiler to change expt/log/softmax to 16b.
Expand All @@ -51,7 +51,7 @@
* **[v0.31.0]**
* **Introduce `quan_config` for `ModelConfig.analysis` for more detailed quantization configuration.**
* **Add `ktc.opt_and_eval` command for quick onnx optimization and evaluation.**
* **Remove deprecated `compilerIpevaluator_730.sh` and add warning messages to other depecated scritps.**
* **Remove deprecated `compilerIpevaluator_730.sh` and add warning messages to other deprecated scripts.**
* Add `compiler_tiling` option for IP evaluator.
* Add `--clear-shapes` and `--replace-avgpool-with-conv` flags to kneronnxopt.
* Add `--seperate` flag to kneronnxopt.onnx_vs_onnx for detailed output comparison.
Expand Down Expand Up @@ -188,7 +188,7 @@
* Support text procssing models.
* Set flatbuffer as the default 720 compiling mode.
* Refactor compiler and analyser inner structure.
* **Due to the structure change, batch compiler do not backwark support previous bie files.**
* **Due to the structure change, batch compiler do not backward support previous bie files.**
* Refactor toolchain manual.
* Bug fixes.
* **[v0.19.0]**
Expand All @@ -198,10 +198,10 @@
* ktc: Add `mode`, `optimize`, `export_dynasty_dump` argument to analysis.
* ktc: Set `skip_verify` in analysis as deprecated.
* regression: Add optimize option for optimization level selection.
* regression: Fix interface to asure platform is integer.
* regression: Fix interface to assure platform is integer.
* converter: Add 720 batch process with `--opt-720` flag.
* converter: Add enable shared weight duplication flag `-d`. By default, shared weights are no longer duplicated.
* converter: Remove `-s` flag since it is now the default behavious.
* converter: Remove `-s` flag since it is now the default behavior.
* converter: Optimize debug output.
* compiler: Fix RDMA not correctly executed.
* E2E simulator: Change dynasty library fetching method.
Expand Down
4 changes: 2 additions & 2 deletions docs/toolchain/appendix/nef_utils_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
POST_ENC : -E(--enc) <nef_file_path> -n(--KN) "kn_number"; [Linux Only]

options:
-o, --output : output file name prefix, defautl is "model_(target)"
-O, --folder : output folder, defautl is "output/"
-o, --output : output file name prefix, default is "model_(target)"
-O, --folder : output folder, default is "output/"
-V, --version : show version number
-H, --help : show this message

Expand Down
6 changes: 3 additions & 3 deletions docs/toolchain/appendix/operators.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,13 +246,13 @@ Notes:
- kernel is kxk & stride is sxs where n and s <= 3 or
- kernel_w & stride_w are 1 and kernel_h & stride_h <= 3 or
- kernel_h & stride_h are 1 and kernel_w = stride_w <= 3)
26. conditions: not pad in batch && any pad in spacial < 32 && constant mode with 0 const_val
26. conditions: not pad in batch && any pad in spatial < 32 && constant mode with 0 const_val
27. conditions: not pad in batch && any of pad < 32 && constant mode with 0 const_val
28. -
29. conditions: power is 2
30. conditions: keepdims = 1
31. contitions: keepdims = 1 && reduce not in batch
32. contitions: keepdims = 1 && reduce in ch
31. conditions: keepdims = 1 && reduce not in batch
32. conditions: keepdims = 1 && reduce in ch
33. -
34. conditions: mode != cubic && extrapolation_value is 0 && rank is 4 && phase_init is {0,0} && nearest_mode is floor if mode is nearest && coordinate_transformation_mode != tf_crop_and_resize
35. conditions: mode != cubic && extrapolation_value is 0 && rank is 4 && not both vus_en and hus_en enabled && phase_init_v >= 0 and delta_v <= 1 if vus_en enabled && phase_init_h >= 0 and delta_h <= 1 if hus_en enabled
Expand Down
2 changes: 1 addition & 1 deletion docs/toolchain/appendix/toolchain_webgui.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ Notes:
## 4. Run IP Evaluator only mode

If you only want to evaluate the model to a estimation of the model performance,
you can turn on the `Run Onnx Flow & IP Evaluator only` switch above the preprocess code seciton.
you can turn on the `Run Onnx Flow & IP Evaluator only` switch above the preprocess code section.
Then you can press run directly. The evaluation result would be shown on the right side `Console Output` section.

## 5. Stop, restart and remove the service
Expand Down
14 changes: 7 additions & 7 deletions docs/toolchain/appendix/yolo_example.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

In this document, we provide a step by step example on how to utilize our tools to compile and test with a newly downloaded YOLOv3 model.

> This document is writen for toolchain v0.30.0. If any description is not consistent with the latest toolchain, please refer to the main toolchain manual.
> This document is written for toolchain v0.30.0. If any description is not consistent with the latest toolchain, please refer to the main toolchain manual.

## Tricks for deploying yolo-type detection models and anker based detection models

Expand All @@ -19,9 +19,9 @@ In this document, we provide a step by step example on how to utilize our tools
2. bbox coordinates with shape 1x4xpixel # at scale 0, 1x4xpixel # at scale 1, ...,1x4xpixel # at scale S
3. Trick: Do NOT concat class scores at different scales. Output class scores for each scale separately.
4. Trick: Do NOT concat class score and coordinates at the same scale. Output class scores and bbox coordinates separately.
5. Trick: Do NOT concat bbox coordinates at differnt scales. Output bbox coordinates for each scale separately.
6. Trick: Typically, class scores need to pass activation fuctions such as exp, sigmoid or even softmax. Make sure these activation fucntions are in the model so that quantiztion algorithm can optimize the quantizaiton setting accordingly.
7. Trick: sometimes, bbox coordinates need to pass exp function or other activation function. Make sure these activation fucntions are in the model so that quantiztion algorithm can optimize the quantizaiton setting accordingly.
5. Trick: Do NOT concat bbox coordinates at different scales. Output bbox coordinates for each scale separately.
6. Trick: Typically, class scores need to pass activation functions such as exp, sigmoid or even softmax. Make sure these activation functions are in the model so that quantiztion algorithm can optimize the quantizaiton setting accordingly.
7. Trick: sometimes, bbox coordinates need to pass exp function or other activation function. Make sure these activation functions are in the model so that quantiztion algorithm can optimize the quantizaiton setting accordingly.
8. Trick: Do NOT concat some outputs and then split in the model. Make sure the computation of all these outputs are separate. If these computation needs to use the same op, the quantization algorithm can detect this situation and share the weights of the same op.


Expand Down Expand Up @@ -61,7 +61,7 @@ python convert.py yolov3-tiny.cfg yolov3-tiny.weights /data1/yolo.h5

We now have `yolo.h5` under our mounted folder `/data1`.

We also need to preprare some images under the mounted folder. We have provided some example input images at <http://doc.kneron.com/docs/toolchain/res/test_image10.zip>.
We also need to prepare some images under the mounted folder. We have provided some example input images at <http://doc.kneron.com/docs/toolchain/res/test_image10.zip>.

Here is how you can get it:

Expand Down Expand Up @@ -112,7 +112,7 @@ from PIL import Image

## Step 2: IP Evaluation

To make sure the onnx model is as expected, we should check the onnx model's performance and see if there are any unsupprted operators (or CPU nodes).
To make sure the onnx model is as expected, we should check the onnx model's performance and see if there are any unsupported operators (or CPU nodes).

```python
# npu (only) performance simulation
Expand Down Expand Up @@ -145,7 +145,7 @@ The estimated FPS (NPU only) report on your terminal should look similar to this
There are two things to take note of in this report:

* Found one CPU node 'KneronResize' in our model
Tthe estimated FPS is 22.5861, the report is for NPU only
The estimated FPS is 22.5861, the report is for NPU only

At the same time, a folder called `compiler` will be generated in your docker mounted folder (`/data1`); the evaluation result will be found in this folder. One important thing is to check the 'ioinfo.csv' in `/data1/compiler`, which looks like this:

Expand Down
8 changes: 4 additions & 4 deletions docs/toolchain/appendix/yolo_example_InModelPreproc_trick.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ In this document, we provide a step by step example on how to utilize our tools
> 1. Step 2: Convert and optimize the pretrain model.
> 2. Step 4: Check ONNX model and preprocess and postprocess are good.

> This document is writen for toolchain v0.22.0. If any description is not consistent with the latest toolchain, please refer to the main toolchain manual.
> This document is written for toolchain v0.22.0. If any description is not consistent with the latest toolchain, please refer to the main toolchain manual.

## Step 0: Prepare environment and data

Expand Down Expand Up @@ -44,7 +44,7 @@ python convert.py yolov3-tiny.cfg yolov3-tiny.weights /data1/yolo.h5

We now have `yolo.h5` under our mounted folder `/data1`.

We also need to preprare some images under the mounted folder. We have provided some example input images at <http://doc.kneron.com/docs/toolchain/res/test_image10.zip>.
We also need to prepare some images under the mounted folder. We have provided some example input images at <http://doc.kneron.com/docs/toolchain/res/test_image10.zip>.

Here is how you can get it:

Expand Down Expand Up @@ -106,7 +106,7 @@ m = ktc.onnx_optimizer.onnx2onnx_flow(m)

Now, in order to make model porting easier, we do In-Model-Preprocess trick. We add a Batchnormalization layer at model front, this Batchnormalization layer will do the following things:
1. divide 255 for every pixel (data normalization required by this model)
2. add 0.5 (hardware require -128 to 127 input but source data is 0 to 255, so we will substract 128 for source data due to hardware requirement, and add back at model's front)
2. add 0.5 (hardware require -128 to 127 input but source data is 0 to 255, so we will subtract 128 for source data due to hardware requirement, and add back at model's front)


```python
Expand All @@ -128,7 +128,7 @@ onnx.save(m, 'yolo.opt.onnx')

## Step 3: IP Evaluation

To make sure the onnx model is as expected, we should check the onnx model's performance and see if there are any unsupprted operators (or CPU nodes).
To make sure the onnx model is as expected, we should check the onnx model's performance and see if there are any unsupported operators (or CPU nodes).

```python
# npu (only) performance simulation
Expand Down
10 changes: 5 additions & 5 deletions docs/toolchain/manual_1_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ In this document, you'll learn:
* Add Einsum defusion in kneronnxopt.
* Support Cast to int64 in knerex and compiler.
* Support HardSwish, Topk and Split nodes in knerex and compiler.
* Update the regression flow log printing. Print success log seperately from errors to avoid confusing.
* Update the regression flow log printing. Print success log separately from errors to avoid confusing.
* Update IP evaluator for DMA with small length.
* Fix the kneronnxopt bug in `replace_Gather_with_Slice`.
* Fix the knerex bug: node Concat channel mismatch.
Expand All @@ -36,7 +36,7 @@ In this document, you'll learn:

In the following parts of this page, you can go through the basic toolchain working process to get familiar with the toolchain.

Below is a breif diagram showing the workflow of how to generate the binary from a floating-point model using the toolchain.
Below is a brief diagram showing the workflow of how to generate the binary from a floating-point model using the toolchain.

<div align="center">
<a href="https://doc.kneron.com/docs/toolchain/imgs/manual/Manual_Flow_Chart.png" target="_blank" rel="noopener noreferrer" data-wm-adjusted="done"><img src="../imgs/manual/Manual_Flow_Chart.png"></a>
Expand Down Expand Up @@ -253,15 +253,15 @@ improve your model performance on-chip waiting in other section. Please check.
* [4. Fixed-Point Model Generation](manual_4_bie.md)
* [5. Compilation](manual_5_nef.md)

There are also other useful tools and informations:
There are also other useful tools and information:

* [End to End Simulator](appendix/app_flow_manual.md): manual for the E2E simualtor.
* [End to End Simulator](appendix/app_flow_manual.md): manual for the E2E simulator.
* [Hardware Performance](appendix/performance.md): common model performance table for Kneron hardwares and the supported operator list.
* [Hardware Supported Operators](appendix/operators.md): operators supported by the hardware.
* [How to Interpret Fixed-Point Report](appendix/fx_report.md): manual for interpreting the fixed-point report.
* [Kneronnxopt](appendix/kneronnxopt.md): manual for the a more flexible onnx optimizer tool.
* [ONNX Converters](appendix/converters.md): manual for the script usage of our converter tools. This tool is only for ONNX opset 11/12.
* [Quantization 1 - Introdution to Post-training Quantization](quantization/1.1_Introdution_to_Post-training_Quantization.md): introduction for the quantization.
* [Quantization 1 - Introduction to Post-training Quantization](quantization/1.1_Introduction_to_Post-training_Quantization.md): introduction for the quantization.
* [Quantization 2 - Post-training Quantization(PTQ) Flow and Steps](quantization/1.2_Flow_and_Steps.md): manual for the quantization flow and steps.
* [Script Tools](appendix/command_line.md): manual for deprecated command line tools. Kept for compatibility of toolchain before v0.15.0)
* [Toolchain History](appendix/history.md): manuals for history version of toolchain and the change log.
Expand Down
Loading
Loading