Added feature for computing codecs chain's strength

Author: dhondta

README.md

@@ -11,7 +11,7 @@
 [![DOI](https://zenodo.org/badge/236679865.svg)](https://zenodo.org/badge/latestdoi/236679865)
 [![License](https://img.shields.io/pypi/l/codext.svg)](https://pypi.python.org/pypi/codext/)
 
-[**CodExt**](https://github.com/dhondta/python-codext) is a library (Python 3.10 or later) that extends the native [`codecs`](https://docs.python.org/3/library/codecs) library (namely for adding new custom encodings and character mappings) and provides **120+ new codecs**, hence its name combining *CODecs EXTension*. It also features a **guess mode** for decoding multiple layers of encoding and **CLI tools** for convenience.
+[**CodExt**](https://github.com/dhondta/python-codext) is a library (Python 3.10 or later) that extends the native [`codecs`](https://docs.python.org/3/library/codecs) library (namely for adding new custom encodings and character mappings) and provides **150+ new codecs**, hence its name combining *CODecs EXTension*. It also features a **guess mode** for decoding multiple layers of encoding and **CLI tools** for convenience.
 
 ```sh
 $ pip install codext

docs/pages/cli.md

@@ -47,6 +47,8 @@ usage examples:
 - echo -en "test" | codext encode upper reverse base32 base64 morse
 - echo -en "test" | codext encode base64 gzip | codext guess
 - echo -en "test" | codext encode base64 gzip | codext guess gzip -c base
+- codext strength base64 gzip
+- echo -en "secret" | codext strength base64 rot-13
 ```
 
 !!! note "Input/output"

docs/pages/guessing.md

@@ -181,3 +181,25 @@ This "textness" depends on the script of the letters. For Latin letters, it is m
 
     This part shall still be improved with better features and/or using machine learning. In particular, the default stop function (`text`) accepts any printable output with a low entropy, hence possibly an intermediate result of a chain of encodings.
 
+### Strength Evaluation
+
+While the guess mode tries to *reverse* an unknown chain of codecs, the strength feature does the opposite: given a chain, it estimates how hard that chain is to reverse. It rates resistance to codext's own guessing heuristic and to casual recognition - not cryptographic strength.
+
+It encodes a sample through the chain, then walks the true decoding path and, at each layer, uses the ranking heuristic (see above) to measure how high the correct codec is ranked. A layer ranked first is an obvious give-away ; a deeply-ranked or non-surfaced ("blind") layer, e.g. one whose codec opts out of guessing, cannot be greedily followed. It also flags an output that still looks like a text (a disguise) and a lossy chain (e.g. a hash) whose plaintext cannot be recovered at all.
+
+The result is a score from 0 (trivially reversible) to 100 (resists the guesser) with a letter grade, the per-layer ranks and notes pointing at the weakest layer:
+
+```python
+>>> codext.strength(["base64"])
+{'chain': ['base64'], 'reversible': True, 'length': 1, 'score': 12, 'grade': 'F', 'blind_layers': 0,
+ 'worst_rank': 1, 'disguised': False, 'correct_ranks': [1],
+ 'notes': ['every layer is ranked first by the guesser: trivially peelable']}
+>>> codext.strength(["gzip", "xor-173", "base64"])["grade"]
+'C'
+```
+
+A compatible sample is chosen automatically ; a specific one can be given with the `sample` keyword-argument. The `include`/`exclude` arguments scope the codecs the guesser may use, exactly like for the guess mode. With the [CLI tool](cli.md), the chain is given as arguments and the sample is read from stdin or a file (a default is used otherwise): `codext strength base64 gzip`.
+
+!!! note "Not cryptographic strength"
+    
+    This measures resistance to codext's guessing/ranking and to casual inspection, not cryptographic security. The per-layer ranking is used on purpose instead of the full recursive guess, which is combinatorially expensive on some chains.

src/codext/VERSION.txt

@@ -1 +1 @@
-2.0.0
+2.0.1

src/codext/__common__.py

@@ -3,6 +3,8 @@
 
 """
 from .__common__ import *
+from .guess import *  # the guess/rank/stopfunc feature (wires codecs.guess and codecs.rank on import)
+from .strength import *  # the strength feature (wires codecs.strength on import)
 from .__info__ import __author__, __copyright__, __email__, __license__, __source__, __version__
 
 

src/codext/__init__.py

@@ -6,6 +6,7 @@
 import sys
 
 from .__common__ import *
+from .guess import stopfunc
 from .__info__ import __author__, __copyright__, __email__, __license__, __source__, __version__
 
 
@@ -151,6 +152,14 @@ def _format_action_invocation(self, action):
                       help="categories, codecs and encodings to be explicitely used ;\n "
                            "format: [category|codec|encoding] OR depth:[category|codec|encoding]")
     rank.add_argument("-l", "--limit", type=int, default=10, help="limit the number of displayed results")
+    strength = sparsers.add_parser("strength", help="evaluate the strength of a chain of codecs", **kw)
+    strength.add_argument("encoding", nargs="+", help="list of encodings forming the chain to evaluate")
+    strength.add_argument("-e", "--exclude", nargs="*", action="extend", metavar="CAT|COD|ENC",
+                          help="categories, codecs and encodings the guessing adversary may not use")
+    strength.add_argument("-f", "--stop-function", default="text", metavar="FUNC",
+                          help="stop function for the guessing adversary (default: text)")
+    strength.add_argument("-i", "--include", nargs="*", action="extend", metavar="CAT|COD|ENC",
+                          help="categories, codecs and encodings the guessing adversary may use")
     search = sparsers.add_parser("search", help="search for codecs")
     search.add_argument("pattern", nargs="+", help="encoding pattern to search")
     listi = sparsers.add_parser("list", help="list items")
@@ -164,7 +173,7 @@ def _format_action_invocation(self, action):
     remm = sparsers.add_parser("remove-macro", help="remove a macro from the registry")
     remm.add_argument("name", help="macro's name")
     args = parser.parse_args()
-    if args.command in ["guess", "rank"]:
+    if args.command in ["guess", "rank", "strength"]:
         args.include, args.exclude = __format_list(args.include), __format_list(args.exclude, False)
     try:
         # if a search pattern is given, only handle it
@@ -181,6 +190,19 @@ def _format_action_invocation(self, action):
         elif args.command == "remove-macro":
             remove_macro(args.name)
             return 0
+        # evaluate the strength of a chain of codecs (sample from a file, from a pipe, or a default one)
+        elif args.command == "strength":
+            sample = None
+            if args.infile:
+                sample = _input(args.infile)
+            elif not sys.stdin.isatty():
+                sample = _input(None)
+            if sample is not None:
+                sample = (sample.rstrip("\r\n") if isinstance(sample, str) else sample.rstrip(b"\r\n")) or None
+            print(codecs.strength_report(args.encoding, sample=sample,
+                                         stop_func=getattr(stopfunc, args.stop_function, stopfunc.default),
+                                         include=args.include, exclude=args.exclude))
+            return 0
         # list encodings or macros
         elif args.command == "list":
             if args.type == "encodings":