RSE API swagger parsing error in API ML

[PavelKondratiev]

Hello team,

I performed static registration of IBM RSE API in Zowe API ML catalog.
When open it in API catalog web interface I get this message :

---

Swagger
API documentation could not be retrieved. There may be something wrong in your Swagger definition.
Please review the values of 'schemes', 'host' and 'basePath'.

---

Here is swagger file in problem
rseapi.swagger.zip

Found related information in this issue #406.
Does that mean API ML works as designed and it is not a bug?

Thanks in advance and best regards

[PavelKondratiev]

Hi team,

Do you have any update on this issue?
Thanks!

[balhar-jakub]

Hi Pavel,

I'm sorry we are swamped with other work right now. It's getting to the top of the queue and I hope that we will get to the topic later this week or beginning of the next one.

[achmelo]

Hi Pavel,
could you please share you static registration file?

[PavelKondratiev]

Hi @achmelo
Here it is

rseapi.zip
Thanks!

[achmelo]

Hi @PavelKondratiev ,
did you import CA certificate of the RSEAPI to the API ML truststore(keyring)?

[PavelKondratiev]

Hi @achmelo

Yes, I did that

Digital ring information for user ZWESVUS2:

  Ring:                                                               
       >ZoweKeyringZOSMF<                                             
  Certificate Label Name             Cert Owner     USAGE      DEFAULT
  --------------------------------   ------------   --------   -------
  zOSMFCA                            CERTAUTH       CERTAUTH     NO   
  ZoweLocalCA                        CERTAUTH       CERTAUTH     NO   
  ZoweCertificate                    ID(ZWESVUS2)   PERSONAL     NO   
  zOSMF_server_Z25B                  ID(IZUSVR)     PERSONAL     YES  
  RSEAPICA                           CERTAUTH       CERTAUTH     NO   
  RSEAPI_server_Z25B                 ID(STCAPI)     PERSONAL     NO   
                                                                      
***                                                                   

In fact I does not think it does matter - I constantly see messages in RSEAPI log

10.1.1.19 - - [12/Jan/2025:00:01:34 -0500] "GET /rseapi/api-docs/swagger.json HTTP/1.1" 200 189409

They tell me Zowe API Ml component is able to read RSEAPI swagger file - isn't it?
Seems the problem is swagger content is unacceptable for Zowe parser.

Thanks

[achmelo]

API ML is able to read RSEAPI swagger file. I used the file you shared with us and I can see the documentation in my API catalog. Did you try to enable debug logs in the Catalog?

[PavelKondratiev]

Hi @achmelo

I assume your API ML is at V3 level - right?
I missed to point my issue occurs in Zowe V2.18.

Do you mean to set components.api-catalog.debug=true or something else?
Thanks

[achmelo]

Yes, that is the correct property for the debug. I will try with v2 and let you know.

[PavelKondratiev]

Hi @achmelo

Btw I get exactly the same error in V3.

I turned on debug for api-catalog as you suggested and its content show me the possible root of the problem - swagger.json contains some tabulation chars :

2025-01-18 16:38:23.804 <ZWEAAC1:https-jsse-nio-0.0.0.0-7552-exec-5:67175061> �[35mZWESVUSR�[0;39m �[36mWARN �[0;39m ((javax.net.ssl)) handling exception
2025-01-18 16:38:30.504 <ZWEAAC1:https-jsse-nio-0.0.0.0-7552-exec-5:67175061> �[35mZWESVUSR�[0;39m �[36mDEBUG�[0;39m ((o.z.a.a.s.s.APIDocRetrievalService)) Retrieving default API doc for service 'ibmrse'
2025-01-18 16:38:31.833 <ZWEAAC1:https-jsse-nio-0.0.0.0-7552-exec-5:67175061> �[35mZWESVUSR�[0;39m �[36mDEBUG�[0;39m ((o.z.a.a.s.s.APIDocRetrievalService)) No API set as default, will use highest major version as default
2025-01-18 16:38:49.369 <ZWEAAC1:https-jsse-nio-0.0.0.0-7552-exec-5:67175061> �[35mZWESVUSR�[0;39m �[36mDEBUG�[0;39m ((o.z.a.a.s.a.ApiTransformationConfig)) Could not convert response body to a Swagger/OpenAPI object.
com.fasterxml.jackson.dataformat.yaml.snakeyaml.error.MarkedYAMLException: while scanning for the next token
found character '\t(TAB)' that cannot start any token. (Do not use \t(TAB) for indentation)
 in 'reader', line 3663, column 11:
              	"$ref": "#/components/parameter ... 
              ^

 at [Source: (StringReader); line: 3661, column: 24]
	at com.fasterxml.jackson.dataformat.yaml.snakeyaml.error.MarkedYAMLException.from(MarkedYAMLException.java:28)
...
2025-01-18 16:38:49.919 <ZWEAAC1:https-jsse-nio-0.0.0.0-7552-exec-5:67175061> �[35mZWESVUSR�[0;39m �[36mDEBUG�[0;39m ((o.z.a.a.s.c.CachedApiDocService)) Exception updating default API doc in cache for 'ibmrse'.
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'abstractApiDocService' defined in class path resource [org/zowe/apiml/apicatalog/swagger/api/ApiTransformationConfig.class]: Failed to instantiate [org.zowe.apiml.apicatalog.swagger.api.AbstractApiDocService]: Factory method 'abstractApiDocService' threw exception with message: Response is not a Swagger or OpenAPI type object.
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:648)
...
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.zowe.apiml.apicatalog.swagger.api.AbstractApiDocService]: Factory method 'abstractApiDocService' threw exception with message: Response is not a Swagger or OpenAPI type object.
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:178)
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:644)
	... 166 common frames omitted
Caused by: jakarta.validation.UnexpectedTypeException: Response is not a Swagger or OpenAPI type object.
	at org.zowe.apiml.apicatalog.swagger.api.ApiTransformationConfig.abstractApiDocService(ApiTransformationConfig.java:60)

2025-01-18 16:38:50.283 <ZWEAAC1:https-jsse-nio-0.0.0.0-7552-exec-5:67175061> �[35mZWESVUSR�[0;39m �[36mERROR�[0;39m ((o.z.a.a.s.c.CachedApiDocService)) No default API doc available for service 'ibmrse'
2025-01-18 16:38:50.316 <ZWEAAC1:https-jsse-nio-0.0.0.0-7552-exec-5:67175061> �[35mZWESVUSR�[0;39m �[36mDEBUG�[0;39m ((o.z.a.a.c.a.ApiCatalogController)) An error occurred when trying to fetch ApiDoc for service: ibmrse, processing can continue but this service will not be able to display any Api Documentation.
Error Message: No API Documentation was retrieved for the service ibmrse.

I assume you tested it on non-z/OS platform, right? It may explain why did not you get this error.
Questions :

1. Last phrase of error message shown in UI looks confusing as it does not correctly address issue's root cause - can it be changed?
2. Is it possible to tolerate these tab chars in Zowe code?

Thanks and regards

[achmelo]

The problem is with the response, I get the same error when I use the JSON file you shared. I had to process it with spring-doc library. What is the server where this API doc is located?

[PavelKondratiev]

RSE API is not an open source tool but rather is a component of IBM proprietary product called z/OS Explorer (Aqua)
So if I understood your question correctly it is not available on github or other public repos.
On my zPDT system it resides in /usr/lpp/IBM/rseapi/tomcat.base/webapps/rseapi/api-docs USS catalog

Hope it helps

[pavel-jares-bcm]

There are multiple things:

1. The source file is a JSON file
   In general, a JSON file could have a mix of indents (spaces, end of lines, spaces). The file itself contains a lot of tabulators (hex 0x09), ie.:

0x20 0x20 0x20 0x20 0x20 0x20 0x09 0x22 0x24 0x72 0x65 0x66 0x22 0x3A 0x20 0x22
"$ref": "

2. The treatment is done by class ApiTransformationConfig
   It parses the source as YAML:

api-layer/api-catalog-services/src/main/java/org/zowe/apiml/apicatalog/swagger/api/ApiTransformationConfig.java

Lines 46 to 47 in 924b132

	ObjectMapper mapper = new ObjectMapper(new YAMLFactory())
	.setSerializationInclusion(JsonInclude.Include.NON_NULL);

3. JSON is basically the subset of YAML
   It is possible to put JSON inside the YAML (or replace it), but there are some restrictions from YAML. One of them is using a tabulator. The tab should be replaced by spaces (usually 2 spaces). But the parser com.fasterxml.jackson.dataformat.yaml.YAMLParser doesn't support it. It just throws an exception.

4. OpenAPI / Swagger parsers choose the right parses
   Ie. io.swagger.v3.parser.OpenAPIV3Parser decides on the beginning to use JSON or YAML parser by the first character ({ at first position means JSON otherwise YAML)

---

The issue is the same for v2 and v3.

The only workaround, for now, is to format YAML differently (ie. do not use pretty print to exclude white spaces).

A potential solution for the issues:

• find a way how to support replacing tabs with spaces (I didn't find a configuration such as this one)
• decide on the beginning which parser to use
• try to read as openApi v3, in case of failing try to read Swagger doc
• (the code is just to detect if the source is OpenApi v3 or Swagger doc) Use a different mechanism to detect the source
• if the first character is { use a stream to reformat JSON

I guess to use multiple parsers depending on the first character looks to me like the most reasonable fix

[PavelKondratiev]

Many thanks for the analysis details provided.
One very last question before closing this topic - do you plan to include this fix\enhancement in product's delivery roadmap\schedule?
Thanks in advance!

[balhar-jakub]

We don't have any plan right now, but as part of our simplification of the API ML architecture we will be changing this area, replacing the implementation and as such it may be resolved as part of this effort.