Update FTP and EDI samples for the latest connector APIs by niveathika · Pull Request #62 · ballerina-guides/integration-samples

niveathika · 2026-08-25T04:16:40Z

Purpose

The ftp-edi-message-to-salesforce-opportunity sample does not compile against current libraries. Two things happened since it was last touched:

EDIFACT D03A renamed its generated surface. ballerinax/edifact.d03a.retail 1.0.1 renamed the record types and fields the sample's transformer is built on, and made several segment fields both optional and nilable.
ballerina/ftp grew convenience APIs (getText, move, getBytes, getJson, …) that make part of the sample's hand-rolled plumbing unnecessary.

While fixing those, three older breakages surfaced that also block compilation — see Approach.

The two EDI samples (edi-code-generation, edi-package-generation) needed no code changes; only their stale distribution values.

Goals

Get all three samples compiling and running against the current distribution, without restructuring them.

Approach

EDIFACT type renames in transformer.bal:

Old	New
`EDI_REQOTE_Request_for_quote_message`	`EDI_REQOTE_REQOTE`
`Segment_group_N_GType`	`Group_N_GType`
`quote.Segment_group_1`	`quote.group_1`
`ref.REFERENCE.REFERENCE`	`ref.Reference?.REFERENCE`
`party.NAME_AND_ADDRESS.Party_function_code_qualifier`	`party.Name_and_address?.PARTY_FUNCTION_CODE_QUALIFIER`
`item.LINE_ITEM.Line_item_identifier`	`item.Line_item?.LINE_ITEM_IDENTIFIER`
`item.QUANTITY` / `QUANTITY_Type`	`item.Quantity` / `Quantity_Type`

Because those fields are now optional and nilable, the nested if guards became early-continue checks.

FTP API adoption:

getText() replaces get() plus a local streamToString() byte accumulator (helper and the ballerina/io import deleted).
move() moves each processed file to ftpProcessedQuotesPath. That configurable was declared but never used, even though the README's Testing step 4 already told users to expect it.

Pre-existing breakage fixed:

transformQuoteRequest was duplicated verbatim in main.bal and transformer.bal — a redeclared symbol error since e9a6b73 (Jan 2024). Kept the transformer.bal copy.
configurable sf:ConnectionConfig now fails with invalid type for configurable variable: expected a subtype of 'anydata', since ConnectionConfig includes *http:ClientConfiguration. Split into salesforceBaseUrl and http:OAuth2RefreshTokenGrantConfig salesforceAuth; the README's Config.toml block is updated to match.
A hardcoded pricebook ID (01s6C000000UN4PQAW) shadowed the salesforcePriceBookId configurable in the PricebookEntry query and as the default on Opportunity.Pricebook2Id. Both now use the configurable.
.devcontainer.json had a trailing comma, making it invalid JSON.

Aligned with the published snippet: the code sample on ballerina.io skips files that
do not end in .edi, but the sample here fed everything in the new-quotes directory
straight to the EDI parser. Added the same guard.

distribution bumped to 2201.13.4 across all three samples, matching the newest samples in this repo.

CI

build-edi was already failing on main at its first build step, so everything after it
had been skipped for a long time:

bal build ./edi-code-generation/edi.bal
-> ballerina: invalid source provided. Please provide a valid Ballerina package,
   workspace or a standalone file.

edi.bal is generated into a directory that has a Ballerina.toml, so it is not a
standalone file. Building the package instead fixes it, and the four steps that had been
skipped (libgen, pack, push, edi_service build) all pass once unblocked.

ftp-edi-message-to-salesforce-opportunity was commented out of the build-packages
matrix, which is precisely why it stayed uncompilable since e9a6b73 without anyone
noticing. It is back in the matrix.

working_with_json and kafka_salesforce_integration hit the same "invalid source
provided" error for the same reason — they are container directories, not packages. The
matrix now points at working_with_json/ballerina,
kafka_salesforce_integration/kafka-message-producer, and
kafka_salesforce_integration/kafka-salesforce-pricebook_update, all of which build clean.

Finally, fail-fast: false on the matrix. One genuine failure
(gdrive-files-to-microsoft-onedrive-files) was cancelling ~30 sibling jobs, so the run
reported a wall of red that said nothing about the samples it claimed to be testing.

Failures left on main, untouched by this PR

These are pre-existing and out of scope here; each needs its own connector migration:

Sample	Cause
`gdrive-files-to-microsoft-onedrive-files`	`DriveItemData` / `uploadFileToFolderByPath` removed in `microsoft.onedrive` 3.0.2
`gcalendar-new-event-to-trello-card`	`Cards` / `addCards` removed in `trello` 2.0.1
`gcalendar-new-event-to-trello-card-and-twillio-msg`	`twilio` 5.0.2 replaced `twilioAuth` with a required `auth`
`hubspot-contacts-to-google-contacts`	`ballerinax/hubspot.crm.contact` no longer resolves
`salesforce-new-contact-to-twilio-sms`	`ballerinax/trigger.salesforce` no longer resolves
`gmail-to-salesforce-lead`	`salesforce` 7.6.0 `NO_CLASS_DEF_FOUND io/ballerina/runtime/api/async/Callback`
`shopify-customer-to-salesforce-customer`	`configurable sf:ConnectionConfig` — same `anydata` breakage fixed here
`servicenow-case-to-salesforce-case`	same as above

The last two would be fixed by the same split this PR applies to main.bal, if a
maintainer wants them folded in.

User stories

As a user following the pre-built integration guides, I can run the FTP/EDI/Salesforce and EDI codegen samples against a current Ballerina distribution.

Release note

Fixed the FTP EDI to Salesforce opportunity sample, which no longer compiled after the EDIFACT D03A type renames and the sf:ConnectionConfig anydata change, and updated it to use the newer ftp:Client getText and move APIs. The sample now also skips non-EDI files in the input directory.

Documentation

The generated-code snippets on the EDI pages under https://ballerina.io/learn/pre-built-integrations/ may need a follow-up: bal edi codegen now additionally emits getSchema(), fromEdiStringWithSchema(), and toEdiStringWithSchema(), and bal edi libgen now also emits rest_connector.bal, transformer.bal, and Module.md. No changes to the sample code are implied.

Training

N/A

Certification

N/A — sample maintenance, no impact on certification exams.

Marketing

N/A

Automation tests

Unit tests

N/A — these samples have no test suite.
Integration tests

Verified manually:
- edi-code-generation: ran bal edi codegen -i schemas/edi-schema.json -o edi.bal, then bal run -- read (prints Order date: 2008-01-01) and bal run -- write (round-trips to the expected HDR*…~ ITM*…~ output).
- edi-package-generation: ran bal edi libgen -p citymart/porder -i schemas -o ., packed and pushed to the local repository, and built edi_service against it.
- ftp-edi-message-to-salesforce-opportunity: bal build clean, bal format --dry-run clean. Parsed resources/inputs/quote1.edi through the updated transformQuoteRequest and confirmed it yields {"oppName":"TS_035","accountName":"TechShop","itemData":[{"itemId":"…","quantity":10}]}.
Not verified end to end: the list/getText/move calls against a live FTP server, and the Salesforce calls. Docker was unavailable in my environment and I have no Salesforce org to point at, so the FTP and Salesforce paths are checked by signature and compilation only. Worth a run by someone with both available before merge.

Security checks

Followed secure coding standards in http://wso2.com/technical-reports/wso2-secure-engineering-guidelines? yes
Ran FindSecurityBugs plugin and verified report? no — not applicable to Ballerina samples.
Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets? yes. This PR in fact removes a hardcoded Salesforce pricebook ID that was committed previously.

Samples

This PR is entirely sample maintenance; see Approach.

Related PRs

None.

Migrations (if applicable)

N/A

Test environment

Ballerina 2201.13.3 (Swan Lake Update 13), bal edi tool 2.2.1, on Linux (WSL2). Dependencies resolved: ballerina/ftp 2.20.2, ballerina/edi 1.6.0, ballerinax/edifact.d03a.retail 1.0.1, ballerinax/salesforce 8.7.0.

Learning

The EDIFACT rename is not documented as a breaking change anywhere I could find; I recovered the new names by reading the type definitions in the cached edifact.d03a.retail 1.0.1 bala directly.

The ftp-edi-message-to-salesforce-opportunity sample no longer compiled. EDIFACT D03A renamed its generated types (EDI_REQOTE_REQOTE, Group_N_GType, lowercase group_N fields) and made several segment fields optional and nilable, so the whole REQOTE transformation was dead code. Take the FTP additions while here: getText() replaces the hand-rolled byte-stream-to-string helper, and move() finally uses ftpProcessedQuotesPath, which was configurable but unused despite the README promising the files get moved. Also fix breakage that predates these library changes: - transformQuoteRequest was duplicated in main.bal and transformer.bal, a redeclared symbol since e9a6b73 - sf:ConnectionConfig is no longer a subtype of anydata, so it cannot be a configurable; split into salesforceBaseUrl and salesforceAuth - a hardcoded pricebook ID shadowed the salesforcePriceBookId configurable in the PricebookEntry query and on Opportunity - .devcontainer.json had a trailing comma Bump distribution to 2201.13.4 across all three samples. The EDI codegen and libgen samples themselves needed no code changes; both were verified against edi tool 2.2.1.

Anything dropped into the new-quotes folder was fed to the EDI parser. The published snippet on ballerina.io already guards this; the sample did not.

build-edi has been failing on main at the first build step: `bal build ./edi-code-generation/edi.bal` points at a file inside a package, which the compiler now rejects with "invalid source provided". Build the package instead. Everything after that step had been skipped, so libgen, pack, push, and the edi_service build had not run in a long time; all four pass once the first step is unblocked. ftp-edi-message-to-salesforce-opportunity was commented out of the build-packages matrix, which is how it stayed broken since e9a6b73. It compiles again, so put it back. Also stop the matrix from cancelling itself. gdrive-files-to-microsoft- onedrive-files fails against microsoft.onedrive 3.0.2 (DriveItemData and uploadFileToFolderByPath are gone), and fail-fast turned that one genuine failure into ~30 cancelled jobs, hiding the real status of every other sample. That sample still needs its own fix.

working_with_json and kafka_salesforce_integration hold sub-projects rather than being packages themselves, so `bal build` on them fails with "invalid source provided" — the same mistake as the edi-code-generation step, not a problem with the samples. All three sub-packages build clean.

niveathika · 2026-08-25T07:34:04Z

Closing this as we need full overhaul of examples

niveathika requested a review from sameerajayasoma as a code owner August 25, 2026 04:16

niveathika added 3 commits August 25, 2026 10:51

Skip non-EDI files in the FTP input directory

5bcf88e

Anything dropped into the new-quotes folder was fed to the EDI parser. The published snippet on ballerina.io already guards this; the sample did not.

niveathika closed this Aug 25, 2026

Navigation Menu

Repository navigation

Update FTP and EDI samples for the latest connector APIs - #62

Update FTP and EDI samples for the latest connector APIs#62
niveathika wants to merge 4 commits into
ballerina-guides:mainfrom
niveathika:update-ftp-edi-samples

niveathika commented Aug 25, 2026 •

edited

Loading

Uh oh!

niveathika commented Aug 25, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

niveathika commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Goals

Approach

CI

Failures left on main, untouched by this PR

User stories

Release note

Documentation

Training

Certification

Marketing

Automation tests

Security checks

Samples

Related PRs

Migrations (if applicable)

Test environment

Learning

Uh oh!

niveathika commented Aug 25, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

niveathika commented Aug 25, 2026 •

edited

Loading