Conversation
|
NB: generated a .md file as well, suitable for DevDocs if desired. |
JamesDeFabia
left a comment
JamesDeFabia
left a comment
There was a problem hiding this comment.
A couple of comments inline
|
|
||
| <sect2 id="DFSClientCerts-Overview"> | ||
| <title>Overview</title> | ||
| <para>By default, dafilesrv pods in HPCC containerized deployments use internal certificates for pod-to-pod encryption. To allow secure access from external clients (e.g., remote ECL IDE, dfuplus, or custom tools), you must configure dafilesrv to require and validate client certificates signed by a trusted CA.</para> |
|
|
||
| <sect2 id="DFSClientCerts-Overview"> | ||
| <title>Overview</title> | ||
| <para>By default, dafilesrv pods in HPCC containerized deployments use internal certificates for pod-to-pod encryption. To allow secure access from external clients (e.g., remote ECL IDE, dfuplus, or custom tools), you must configure dafilesrv to require and validate client certificates signed by a trusted CA.</para> |
There was a problem hiding this comment.
Certificate Authority (CA)
|
Jira Issue: https://hpccsystems.atlassian.net//browse/HPCC-27274 Jirabot Action Result: |
Signed-off-by: Panagiotatos <greg.panagiotatos+copilot@lexisnexisrisk.com>
|
|
||
| <sect2 id="DFSClientCerts-ClientConfig"> | ||
| <title>Step 4: Configure External Clients</title> | ||
| <para>Configure your external client (e.g., dfuplus, ECL IDE, or custom application) to use the client certificate and key, and to trust the CA certificate. For example, with <emphasis>dfuplus</emphasis>:</para> |
There was a problem hiding this comment.
For consistency, capitalize DFUPlus like line 866
jakesmith
left a comment
jakesmith
left a comment
There was a problem hiding this comment.
@g-pan - see a few comments, but I for one am confused by these steps and what it's trying to document.
The process of setting up certificate issues in a k8s setup at least, does not involve manually defining manifest like this, it is handed by the cert-manager.
This README is relevant: https://github.com/hpcc-systems/HPCC-Platform/blob/master/helm/examples/certmanager/README.md
This document is under ContainerizedHPCC.. but what are the goals?
There is typically no need to access the DFS service directly unless it is inconnectnig with another environment .. (but docs mention dfuplus, ECL IDE .. I'm not sure of their relevance).
If inter environment DFS connectivity, is it k8s to k8s, bm->k8s, k8s->bm, all of the those.
There are different steps involved.
This is relevant for DFS from BM: https://hpccsystems.atlassian.net/browse/HPCC-30412 (and associated step-by-step notes in https://hpccsystems.atlassian.net/browse/HPCC-27688)
|
|
||
| <sect2 id="DFSClientCerts-Overview"> | ||
| <title>Overview</title> | ||
| <para>By default, dafilesrv pods in HPCC containerized deployments use internal certificates for pod-to-pod encryption. To allow secure access from external clients (e.g., remote ECL IDE, DFUPlus, or custom tools), you must configure dafilesrv to require and validate client certificates signed by a trusted Certificate Authority (CA).</para> |
There was a problem hiding this comment.
this feels confusing .. why does ECL IDE or DFUPLus .. need direct physical access to dafilesrv's (direct-access version in cloud deployment) ?
There was a problem hiding this comment.
@jakesmith - It seems reasonable, suggested by copilot
There was a problem hiding this comment.
The ECL IDE or DFUPlus will never directly "talk to the data" they will get their data via an esp service, so there is configuration where they access dafilesrv.
There was a problem hiding this comment.
By default, dafilesrv pods in HPCC containerized deployments use internal certificates for pod-to-pod encryption.
This is misleading. Normally there is no dafilesrv involved at all. Normally a containerized delpoyment uses PVC+mounts - not dafilesrv.
dafilesrv serves 3 functions in a cloud environment:
- rowservice - primarily intended for streaming connectors like java/spark
- direct-access - for external systems, primarily BM systems that can't mount the cloud storage directly
- spray-service - used internally a service used by DFUServer to spray/despray.
None of these are normally used when reading/writing logical files.
|
|
||
| <sect1 id="DFSClientCerts" role="nobrk"> | ||
| <title>Configuring DFS Client Certificates for Secure External Access</title> | ||
| <para>This section describes how to configure HPCC Systems<superscript>®</superscript> to require and validate client certificates for external DFS (dafilesrv) access, using Kubernetes, cert-manager, and mTLS.</para> |
There was a problem hiding this comment.
for external DFS (dafilesrv) access
that's misleading.. DFS isn't just dafilesrv .. DFS is logical file access. The physical data may be accessed via mounts and storge accounts (it is typically in cloud envs)..
There was a problem hiding this comment.
@jakesmith - is this statement in the docs incorrect?
There was a problem hiding this comment.
as I say, it's misleading. You don't typically access remote data via dafilesrv. All our cloud systems access other clouds sysytems via ~remote that uses DFS, but access the physical data via mount points to PVC's - not dafilesrv.
dafilesrv is an option, and the preferred option at the moment for BM systems accessing DFS.
Medium term, everything will switch to using Azure API for data access, client systems will continue to use DFS, but physical data will not use dafilesrv or PVC mount points, but Azure API for access to the storage accounts.
|
|
||
| <sect2 id="DFSClientCerts-GenerateCA"> | ||
| <title>Step 1: Create a Certificate Authority (CA)</title> | ||
| <para>Use <emphasis role="strong">cert-manager</emphasis> to create a CA in your Kubernetes cluster. For example:</para> |
There was a problem hiding this comment.
this process is not familiar to me...
I think part of the confusion (mine too) is .. is this trying to document the setup for Bare Metal, for a k8s setup, for both at same time?
There was a problem hiding this comment.
@jakesmith - unsure, suggested by copilot, seemed reasonable
Signed-off-by: Panagiotatos <greg.panagiotatos+copilot@lexisnexisrisk.com>
|
Apologies @jakesmith, I had copilot analyze the JIRA and the parent JIRA and pull request in crafting this and don't understand the topic well enough to question copilots suggestions, or reply to your questions. |
| <listitem><para>Kubernetes to bare metal (k8s-to-bm)</para></listitem> | ||
| <listitem><para>Legacy tooling or remote clients (e.g., dfuplus, ECL IDE) accessing DFS from outside the cluster</para></listitem> | ||
| </itemizedlist> | ||
| For standard single Kubernetes cluster deployments, direct DFS access is rarely needed; internal pod-to-pod encryption is handled automatically. |
There was a problem hiding this comment.
pod-to-pod encryptiion - whether present or not, is not related to this topic.
External accesss (via DFS service) is about a separate HPCC deployment (could be a on k8s, could be on BM), accessing the remote data. For that it (the client environments) should use ~remote:: ... and the DFS service, and it should be secure with client certificates.
| </para> | ||
|
|
||
| <para> | ||
| <emphasis role="bold">dafilesrv:</emphasis> dafilesrv is one mechanism for accessing the physical data behind logical DFS files, especially for remote, hybrid, or inter-environment scenarios. Secure access to dafilesrv is required when clients or tools (such as ECL IDE, DFUPlus, or custom applications) need to access DFS files from outside the cluster or across environments. |
There was a problem hiding this comment.
The ECL IDE or DFUPlus will never directly "talk to the data" they will get their data via an esp service, so there is configuration where they access dafilesrv.
|
|
||
| <sect2 id="DFSClientCerts-Overview"> | ||
| <title>Overview</title> | ||
| <para>By default, dafilesrv pods in HPCC containerized deployments use internal certificates for pod-to-pod encryption. To allow secure access from external clients (e.g., remote ECL IDE, DFUPlus, or custom tools), you must configure dafilesrv to require and validate client certificates signed by a trusted Certificate Authority (CA).</para> |
There was a problem hiding this comment.
The ECL IDE or DFUPlus will never directly "talk to the data" they will get their data via an esp service, so there is configuration where they access dafilesrv.
|
|
||
| <sect2 id="DFSClientCerts-Overview"> | ||
| <title>Overview</title> | ||
| <para>By default, dafilesrv pods in HPCC containerized deployments use internal certificates for pod-to-pod encryption. To allow secure access from external clients (e.g., remote ECL IDE, DFUPlus, or custom tools), you must configure dafilesrv to require and validate client certificates signed by a trusted Certificate Authority (CA).</para> |
There was a problem hiding this comment.
By default, dafilesrv pods in HPCC containerized deployments use internal certificates for pod-to-pod encryption.
This is misleading. Normally there is no dafilesrv involved at all. Normally a containerized delpoyment uses PVC+mounts - not dafilesrv.
dafilesrv serves 3 functions in a cloud environment:
- rowservice - primarily intended for streaming connectors like java/spark
- direct-access - for external systems, primarily BM systems that can't mount the cloud storage directly
- spray-service - used internally a service used by DFUServer to spray/despray.
None of these are normally used when reading/writing logical files.
Signed-off-by: Panagiotatos <greg.panagiotatos+copilot@lexisnexisrisk.com>
🔄 Upmerge Test ResultsStatus: ✅ All branches merged successfully ✅ Successful Branches (1)
|
|
@jakesmith hopefully this better addressed your concerns |
| </para> | ||
|
|
||
| <para> | ||
| <emphasis role="bold">Certificate Management:</emphasis> In Kubernetes, certificate management is typically automated using <emphasis>cert-manager</emphasis>. Manual manifest examples are provided for reference and advanced use cases, but most users should follow automated workflows. See the <ulink url="https://github.com/hpcc-systems/HPCC-Platform/blob/master/helm/examples/certmanager/README.md">cert-manager README</ulink> for best practices. |
There was a problem hiding this comment.
The URL link does not work in the rendered PDF
There was a problem hiding this comment.
Has this been resolved?
|
|
||
| <para> | ||
| <emphasis role="bold">Further Reading:</emphasis> For bare metal DFS connectivity and step-by-step hybrid setup notes, see <ulink url="https://hpccsystems.atlassian.net/browse/HPCC-30412">HPCC-30412</ulink> and <ulink url="https://hpccsystems.atlassian.net/browse/HPCC-27688">HPCC-27688</ulink>. | ||
| </para> |
There was a problem hiding this comment.
The URL link does not work in the rendered PDF.
But more importantly, linking to a Jira issue is generally not a good idea.
Signed-off-by: Panagiotatos <greg.panagiotatos+copilot@lexisnexisrisk.com>
🔄 Upmerge Test ResultsStatus: ✅ All branches merged successfully ✅ Successful Branches (1)
|
Signed-off-by: Panagiotatos <greg.panagiotatos+copilot@lexisnexisrisk.com>
🔄 Upmerge Test ResultsStatus: ✅ All branches merged successfully ✅ Successful Branches (1)
|
JamesDeFabia
left a comment
JamesDeFabia
left a comment
There was a problem hiding this comment.
One URL issue remains
| </para> | ||
|
|
||
| <para> | ||
| <emphasis role="bold">Certificate Management:</emphasis> In Kubernetes, certificate management is typically automated using <emphasis>cert-manager</emphasis>. Manual manifest examples are provided for reference and advanced use cases, but most users should follow automated workflows. See the <ulink url="https://github.com/hpcc-systems/HPCC-Platform/blob/master/helm/examples/certmanager/README.md">cert-manager README</ulink> for best practices. |
There was a problem hiding this comment.
Has this been resolved?
Signed-off-by: Panagiotatos <greg.panagiotatos+copilot@lexisnexisrisk.com>
🔄 Upmerge Test ResultsStatus: ✅ All branches merged successfully ✅ Successful Branches (1)
|
| <emphasis>cert-manager</emphasis>. Manual manifest examples are provided | ||
| for reference and advanced use cases, but most users should follow | ||
| automated workflows. See the <ulink | ||
| url="https://github.com/hpcc-systems/HPCC-Platform/blob/master/helm/examples/certmanager/README.md">https://github.com/hpcc-systems/HPCC-Platform/blob/master/helm/examples/certmanager/README.md |
There was a problem hiding this comment.
The link in the PDF takes me to
https://github.com/hpcc-systems/HPCC-Platform/tree/master/helm
instead of
https://github.com/hpcc-systems/HPCC-Platform/blob/master/helm/examples/certmanager/README.md
Signed-off-by: Panagiotatos <greg.panagiotatos+copilot@lexisnexisrisk.com>
🔄 Upmerge Test ResultsStatus: ✅ All branches merged successfully ✅ Successful Branches (1)
|
jakesmith
left a comment
jakesmith
left a comment
There was a problem hiding this comment.
@g-pan - I set copilot to review, and cross-reference with codebase and push comments.
I've looked through the comments, deleted any that were out of scope, or irrelevant, I think the thrust of it's comments are valid though.
The doc. is still a bit out of kilter with the helm chart / values approach etc.
- Notes on certificate manager : https://github.com/hpcc-systems/HPCC-Platform/blob/master/helm/examples/certmanager/README.md
- Notes on DFS (not to be conflated with dafilesrv) cert setup for a remote DFS system in a k8s/containized client : https://hpccsystems.atlassian.net/browse/HPCC-27274 (the doc attached there).
- Notes on steps needed for prep to setup BM client to a remote DFS: https://hpccsystems.atlassian.net/browse/HPCC-33260 (and link to https://hpccsystems.atlassian.net/browse/HPCC-27688 for BM remote side config)
| enabled: true | ||
| secret: dafilesrv-server-tls | ||
| caCertSecret: dfs-ca-root | ||
| </programlisting> |
There was a problem hiding this comment.
bug: Fabricated Helm values structure — The dafilesrv.tls block with keys enabled, secret, caCertSecret does not exist in the HPCC Helm chart (helm/hpcc/values.yaml).
The actual mechanism for dafilesrv TLS is:
- The global
certificatessection invalues.yamlwith issuers (local,public,remote,signing) certificates.enabled: trueactivates cert-manager integration- The dafilesrv Helm template (
dafilesrv.yaml) picks the issuer based on service visibility
These fabricated values would be silently ignored by Helm. The documentation should reference the actual certificates section and the trustClients mechanism that dafilesrv supports (see dafilesrv.yaml:113 in the template).
| dfuplus ... | ||
| -tlsclientcert=alice.crt \ | ||
| -tlsclientkey=alice.key \ | ||
| -tlscacert=ca.crt ... |
There was a problem hiding this comment.
bug: Fabricated DFUPlus CLI flags — The flags -tlsclientcert, -tlsclientkey, -tlscacert do not exist in the DFUPlus codebase (dali/dfuplus/dfuplus.cpp).
DFUPlus uses mtls-secret=<secretName> to reference a Kubernetes secret for mTLS (see dfuplus.cpp:357). Furthermore, the overview section of this document correctly states that "DFUPlus and ECL IDE do not directly access dafilesrv" — they communicate with ESP services. Showing DFUPlus as a dafilesrv client contradicts that statement.
This entire step should either be removed or rewritten to document the actual mtls-secret mechanism for ESP communication.
| </para> | ||
| </sect2> | ||
|
|
||
| <sect2 id="DFSClientCerts-GenerateCA"> |
There was a problem hiding this comment.
bug: Manual cert-manager resources conflict with HPCC's integrated approach — These steps show manually creating a ClusterIssuer and CA Certificate. However, HPCC's Helm chart has integrated cert-manager support via the certificates.issuers section in values.yaml (lines 320-380), which auto-creates issuers and certificates through Helm templates.
The manual approach documented here would either duplicate or conflict with the chart-managed resources. Users should be directed to:
- Set
certificates.enabled: truein their values override - Configure the appropriate issuer (e.g.,
certificates.issuers.remote) - Use
trustClientson dafilesrv entries for client cert trust
See helm/examples/certmanager/README.md for the intended workflow.
|
|
||
| <para> | ||
| <emphasis role="bold">Note:</emphasis> DFUPlus and ECL IDE do not directly access dafilesrv; they interact with HPCC Systems<superscript>®</superscript> via ESP services. Client certificates for dafilesrv are only needed for secure external access in hybrid/inter-environment scenarios. | ||
| </para> |
There was a problem hiding this comment.
discuss: DFS vs dafilesrv scope confusion — "DFS client certificates" conflates DFS (the logical file access layer) with dafilesrv (one specific physical access mechanism). As the Overview section correctly notes, DFS is about logical file access. Client certificates here are specifically about securing dafilesrv connections, not DFS in general.
Suggested title: "Configuring Client Certificates for Secure dafilesrv Access" or similar. (Also raised by jakesmith in earlier review.)
| dafilesrv: | ||
| tls: | ||
| enabled: true | ||
| requireClientCert: true |
There was a problem hiding this comment.
bug: Fabricated Helm values (duplicate) — Same issue as Step 2: dafilesrv.tls.requireClientCert does not exist in the Helm chart. The actual mechanism for requiring client certificates on dafilesrv is via trustClients on the dafilesrv entry in values.yaml, combined with enabling certificates.issuers.remote. See helm/hpcc/templates/dafilesrv.yaml:113 which passes trustClients to hpcc.addCertificate.
| </sect2> | ||
| </sect1> | ||
|
|
||
| <sect1 id="DFSClientCerts" role="nobrk"> |
There was a problem hiding this comment.
discuss: Overall concern — AI-generated content not validated against codebase. Multiple review comments from the author indicate this content was "suggested by Copilot, seemed reasonable." However, the Helm values, CLI flags, and cert-manager workflow are all fabricated — they don't match the actual code. The real mechanisms to document are:
certificatessection invalues.yaml(global cert-manager integration)trustClientson dafilesrv/ESP/Roxie entries (client cert trust)mtls-secretfor DFUPlus ESP communication
I'd recommend rewriting based on the actual helm/hpcc/values.yaml (lines 320-380, 478-505) and helm/examples/certmanager/README.md.
Type of change:
Checklist:
Smoketest:
Testing:
Successful Unit Testing: https://github.com/g-pan/github-action-dev-build/actions/runs/21866214565