Skip to content

Add OCI NoSQL Database: tables, rows, indexes - #424

Open
arunesh-j wants to merge 1 commit into
developmentfrom
feat/oci-nosql
Open

Add OCI NoSQL Database: tables, rows, indexes#424
arunesh-j wants to merge 1 commit into
developmentfrom
feat/oci-nosql

Conversation

@arunesh-j

Copy link
Copy Markdown
Collaborator

Summary

  • Implements OCI NoSQL Database Cloud Service against the existing portable database driver.
  • Includes a real DDL parser, since OCI's CreateTable takes a SQL-ish statement rather than an attribute list.
  • Nothing added to services/database/driver; OCI-only behaviour is a consumer-side Extras interface, per Move OCI-only capabilities out of shared driver packages #393.

Closes #412. Part of #376.

Changes

  • providers/oci/nosql/Mock over memstore implementing driver.Database, guarded by a sync.RWMutex, plus the DDL parser and a query evaluator.
  • server/oci/nosql/ — the /20190828/ surface.
  • Wiring is one line each in providers/oci/oci.go and server/oci/oci.go.

Operations: tables (Create/List/Get/Update/Delete/ChangeCompartment), indexes (Create/List/Get/Delete), rows (GetRow/UpdateRow/DeleteRow), and Query. All 24 portable Database methods implemented.

DDL — what is parsed, what is refused

Parsed: CREATE TABLE and ALTER TABLE — scalar and JSON types, NOT NULL, DEFAULT, PRIMARY KEY with optional SHARD, USING TTL <n> DAYS, ADD/DROP on non-key columns, IF NOT EXISTS.

Refused by name, never silently accepted:

Refused Why
Primary keys wider than two columns, composite shard keys The portable partition/sort pair cannot identify such a row. This is a correctness rejection, not a shortcut
ARRAY / MAP / RECORD / ENUM Not modelled
generated / MR_COUNTER / UUID modifiers Not modelled
USING TTL … HOURS OCI's own Schema reports TTL only in days
MODIFY, schema freezing, JSON-path index keys Not modelled
Query projections, aggregates, joins, ORDER BY, range conditions Not modelled

Judgement calls

  • No change streams. OCI publishes no DynamoDB-Streams equivalent, so UpdateStreamConfig/GetStreamRecords return Unimplemented rather than an empty iterator. Noted in docs/services.md.
  • Multi-delete is DELETE FROM … WHERE over /query — OCI's REST API has no MultiDelete operation; this is the real mechanism.
  • ListIndexes requires compartmentId although real OCI marks it optional, so no list is ever unscoped. Documented.
  • 501, named, for /tables/{id}/usage and /query/prepare, /query/summarize. usage is omitted from row and query responses for the same reason — zeros would read as real telemetry.

Provider Coverage

  • AWS
  • Azure
  • GCP
  • OCI

Checklist

  • All tests pass (go test ./...) — exit 0, 272 packages
  • Linter passes (golangci-lint run --timeout=9m) — 0 issues
  • Every provider the change applies to implements the same behavior — OCI-only, additive
  • Integration tests added to cloudemu_test.go — driver + handler tests instead
  • Unit tests added to provider test files

Test Plan

go build ./...                                              clean
go test ./...                                               exit 0, 272 packages
go test -race ./providers/oci/... ./server/oci/...          11/11 ok
golangci-lint run --timeout=9m ./providers/oci/... ./server/oci/...   0 issues
go generate ./...                                           docs/coverage committed

Coverage leak check clean: no OCI operation in docs/coverage/{aws,azure,gcp}/*.md; git diff development -- services/ empty.

End-to-end on a running server (port 4613):

CreateTable                        -> 202 + Opc-Work-Request-Id
GetTable                           -> schema, primaryKey [id,email], shardKey [id], ttl 0
UpdateRow / GetRow                 -> 200 / value round-trips
ListTables                         -> 1 table; other compartment -> []; no compartmentId -> 400
CreateIndex                        -> 202 + work request; index ACTIVE
SELECT * FROM users                -> 3 rows
DELETE FROM users WHERE id = 2     -> NumRowsDeleted 2
DeleteRow twice                    -> 200, then isSuccess false
TRUNCATE TABLE                     -> 400 naming "TRUNCATE TABLE"
3-column PRIMARY KEY               -> 400 naming the limit
ARRAY(STRING)                      -> 400 naming "ARRAY"
USING TTL 6 HOURS                  -> 400 naming "HOURS"
ON_DEMAND + maxReadUnits           -> 400
undeclared column                  -> 400
SELECT name FROM users             -> 400 naming projections
GET /tables/users/usage            -> 501
DeleteTable                        -> 202; GetTable -> 404; ListTables -> []
USING TTL 7 DAYS                   -> schema.ttl 7; row carries timeOfExpiration

A note on parallel worktrees

An earlier run of this branch failed cmd/cloudemu TestServeOutOfProcess. It is not this change — cmd/cloudemu is untouched by the diff. Six Wave 2 worktrees were running their suites concurrently and that test contends on the shared ~/.cloudemu daemon lock. Verified: with nothing else running it passes, and the full suite is exit 0. Worth knowing as shared-state fragility whenever suites run in parallel.

Implements OCI NoSQL Database against the portable database driver:
providers/oci/nosql holds the mock over memstore, server/oci/nosql the
/20190828 wire handler.

Tables are created from a DDL statement rather than a key list, so the
provider parses CREATE TABLE and ALTER TABLE — scalar and JSON column
types, NOT NULL, DEFAULT, PRIMARY KEY with an optional SHARD, USING TTL
in days, and ADD/DROP on a non-key column. Everything else is refused
with the construct named: primary keys wider than the portable
partition/sort key pair, composite shard keys, structured column types,
generated and MR_COUNTER modifiers, TTL in hours, MODIFY and schema
freezing, and JSON-path index keys.

Tables carry an OCID, a compartment recorded at create and capacity
limits validated against their mode; both list routes require
compartmentId. Table and index mutations record a work request and stamp
opc-work-request-id. Rows are addressed by typed primary key columns and
written synchronously.

The query endpoint runs SELECT * and DELETE FROM with AND-ed equality
conditions — the REST API has no MultiDelete, so DELETE FROM ... WHERE is
the multi-row delete. Table usage and the prepared-statement endpoints
answer 501 naming the gap. OCI NoSQL publishes no change stream, so the
portable stream operations report Unimplemented rather than an empty
iterator.

The OCI-only surface is a consumer-side Extras interface in
server/oci/nosql with its value types in providers/oci/nosql; nothing is
added to services/database/driver.

Closes #412
@arunesh-j arunesh-j added the oci Oracle Cloud Infrastructure label Aug 21, 2026

@NitinKumar004 NitinKumar004 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review notes

Real data-plane engine (per #427): N/A.

Findings

Medium · real-engine — Table names are globally unique, not per-compartment
providers/oci/nosql/nosql.go:417
If a user creates table "orders" in compartment A and then "orders" in compartment B -> the second CreateTable returns 409 AlreadyExists, because table names are a global key rather than compartment-scoped; real OCI scopes NoSQL table names per compartment and both creates succeed, so a multi-compartment SDK/CLI test that reuses a table name across compartments breaks against the emulator only.

m.tables is keyed by table name globally (nosql.go:175); newTable rejects with AlreadyExists on m.tables.Has(name) (nosql.go:417), and CreateOCITable checks m.tables.Get(d.Table) irrespective of spec.CompartmentID (table_extras.go:38-44). resolve()/lookup() also address tables by name globally.

Low · coverage — Portable Query/Scan operators and typed coercion below the 90% coverage pillar
providers/oci/nosql/rows.go:324
If a user issues a portable Query/Scan with a relational sort condition (>, BETWEEN) or stores a LONG/FLOAT/BOOLEAN-typed row -> the ordering (compareOrdering/compareStrings) and coercion (parseTyped/convertTyped) execute for the first time in production with no test guarding them, so a comparison- or coercion-logic regression would ship undetected.

go test -cover: provider 85.8%, server 83.4% (both < 90%). compareOrdering (rows.go:324) 0%, compareStrings (rows.go:341) 0%, compareOp (rows.go:306) 28.6%, parseTyped (row_extras.go:211) 35.7%, convertTyped (row_extras.go:242) 36.8%, SetMonitoring (nosql.go:194) 0%. Query tests exercise only SortOp "="; <, >, <=, >=, BETWEEN and CONTAINS/BEGINS_WITH ordering are never run.

Low · wire-fidelity — No oci-go-sdk SDK-compat test for the handler
server/oci/nosql/handler_test.go:94
If the real oci-go-sdk nosql client shapes a request/response field differently than the hand-modeled types.go structs (e.g. a body vs query-parameter for the Query limit, or a header the SDK requires) -> the divergence is not caught by the current tests, so a genuine client could fail at a step the httptest cases pass; adding one SDK create/get/query round-trip would close this.

handler_test.go drives the handler via raw httptest requests only; the only reference to github.com/oracle/oci-go-sdk is the package doc comment in handler.go. oci-conventions.md calls an SDK round-trip against httptest.NewServer "the strongest evidence the handler is right" (recommended, not required).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

oci Oracle Cloud Infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OCI NoSQL Database: tables, rows, indexes

2 participants