Skip to content

feat postgre: Structure slicing - #1365

Open
zmij wants to merge 6 commits into
userver-framework:developfrom
zmij:feat/pg-struct-view
Open

zmij wants to merge 6 commits into
userver-framework:developfrom
zmij:feat/pg-struct-view

Conversation

@zmij

@zmij zmij commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add StructView<T, Members...> for reading and writing a selected subset of fields of a C++ structure.

This allows application structures to be used directly with PostgreSQL even when a query operates on only part of the structure. The selected members are specified at compile time using member pointers.

The same view can be used for both reading and writing:

using StoredUser = StructView<
    User,
    &User::id,
    &User::name,
    &User::email>;

auto user = result.AsSingleRow<StoredUser>(kRowTag);

transaction.Execute(
    "UPDATE users SET name = $1, email = $2 WHERE id = $3",
    StructView<User, &User::name, &User::email, &User::id>{user});

With no members specified, StructView<T> represents the complete structure using Boost.PFR.

Reading stored fields into a larger application structure

Application structures often contain fields that are not stored in a particular database table. Previously, direct row conversion required the PostgreSQL row to match the complete structure.

For example:

struct User {
    std::int64_t id;
    std::string name;
    std::string email;

    // Computed or populated elsewhere
    Permissions permissions;
    std::vector<Team> teams;
};

while the database stores only:

SELECT id, name, email
FROM users

The stored projection can now be described directly:

using StoredUser = StructView<
    User,
    &User::id,
    &User::name,
    &User::email>;

auto users = result.AsContainer<std::vector<StoredUser>>(kRowTag);  // std::vector<User>

The selected fields are populated from the PostgreSQL row, while the remaining members keep their default values. A container of views is rebound to a container of the underlying structure, allocator included.

This avoids introducing database-specific DTOs or manually copying query results into the application's existing types.

Partial updates

The same mechanism can select exactly the members used as query arguments:

using UserNameUpdate = StructView<
    User,
    &User::name,
    &User::email,
    &User::id>;

transaction.Execute(
    "UPDATE users "
    "SET name = $1, email = $2 "
    "WHERE id = $3",
    UserNameUpdate{user});

The structure remains the source of the arguments, while the view explicitly defines their order and which members participate in the query.

Ad-hoc projections can be used directly when introducing a named type is unnecessary:

transaction.Execute(
    "UPDATE users SET name = $1 WHERE id = $2",
    StructView<User, &User::name, &User::id>{user});

MakeStructView deduces the structure from the argument, so only the members have to be written out:

transaction.Execute(
    "UPDATE users SET name = $1 WHERE id = $2",
    MakeStructView<&User::name, &User::id>(user));

Queries using a single structure member

Views are also useful when only one member of an existing object is required:

transaction.Execute(
    "DELETE FROM users WHERE id = $1",
    StructView<User, &User::id>{user});

There is no need to extract user.id separately or introduce another argument representation.

Full structures

An empty member list means the complete structure:

StructView<User>

and uses Boost.PFR structural introspection.

This makes full and partial structural mappings use the same abstraction:

StructView<User>                         // all fields
StructView<User, &User::id>              // id only
StructView<User, &User::name, &User::id> // selected fields

Decomposed and bulk execution of containers

ExecuteDecompose and ExecuteDecomposeBulk send a container of structures column by column, one array parameter per member. A view now selects which members become those columns, and in which order. The container keeps holding the original structures, and the view is applied to each element:

std::vector<User> users = ...;

// The view type named explicitly
using UserIdName = StructView<User, &User::id, &User::name>;
transaction.ExecuteDecompose(
    "INSERT INTO users(id, name) SELECT * FROM unnest($1, $2)",
    StructViewContainer<UserIdName>(users));

// The structure deduced from the container, only the members named
transaction.ExecuteDecomposeBulk(
    "INSERT INTO users(email, id) SELECT * FROM unnest($1, $2)",
    MakeViewContainer<&User::email, &User::id>(users),
    chunk_rows);

// Every member, through the cluster
cluster.ExecuteDecompose(
    ClusterHostType::kMaster,
    "SELECT * FROM unnest($1, $2, $3)",
    MakeViewContainer<>(users));

Supported calls:

Call Transaction Cluster
ExecuteDecompose (with and without command control) yes yes
ExecuteDecomposeBulk (with and without command control) yes not provided
ExecuteBulk rejected at compile time not provided
  • StructViewContainer<View>(container) and MakeViewContainer<Members...>(container) are helper functions returning a borrowing wrapper, so the wrapper must not outlive the container. C++20 class template argument deduction can't take the view explicitly and also deduce the container, which is why these are functions.
  • ExecuteBulk sends each chunk as a single array of the element type. A subset of a structure's members has no PostgreSQL type to map to, so passing a view container fails to compile with a hint to use ExecuteDecomposeBulk.
  • Cluster has no bulk variants, as before: bulk runs several statements and needs a transaction around them.

The column-splitting helpers now project through the view, with StructView<value_type> as the default. As a result, plain ExecuteDecompose and ExecuteDecomposeBulk also accept containers of structures with an intrusive Introspect(). Previously they were limited to Boost.PFR aggregates.

Motivation

Queries rarely map one-to-one to complete application structures. Application types may contain computed or transient data, while updates and deletes often operate on only a handful of fields.

StructView makes these projections explicit without requiring:

  • query-specific DTOs;
  • manual extraction of individual structure members;
  • intermediate JSON or other serialization representations;
  • runtime field-name mappings.

Member pointers keep the mapping compile-time checked and refactoring-friendly, while Boost.PFR provides the full-structure case without additional declarations.


Note: by creating a PR or an issue you automatically agree to the CLA. See CONTRIBUTING.md. Feel free to remove this note, the agreement holds.

@zmij
zmij marked this pull request as ready for review October 8, 2026 08:28
@zmij zmij changed the title [feat] PostgreSQL: Structure slicing feat postgre: Structure slicing Oct 8, 2026
Comment thread postgresql/include/userver/storages/postgres/io/struct_view.hpp Outdated
zmij added 2 commits October 9, 2026 04:36
… the standard library

Rebinding a container and its allocator is not a postgres concern, and the
allocator half was hand-rolled where the standard library already answers it.
Both move to meta::RebindContainer in userver/utils/meta.hpp, beside the other
container traits, and an allocator is now rebound through
std::allocator_traits<Alloc>::rebind_alloc<T>.

Writing a test for it found the shape was wrong for associative containers.
std::set<int> is set<int, less<int>, allocator<int>>, and under P0522 that binds
a two-parameter Container<OldT, Alloc> specialisation as
Container<int, std::less<int>> -- so it rebound the comparator and dropped the
allocator. The old code only looked correct because substituting
less<int> -> less<string> left the allocator to default to the right one; given
a non-default allocator it discarded it silently.

So the trait now takes the whole type-argument list: an allocator for the old
value type is rebound through std::allocator_traits, a policy parameterised by
that type (std::less, std::hash, std::equal_to) is substituted, and anything
else is carried across untouched. The auto... form for std::array is unchanged.

TEST(Meta, RebindContainer) carries a control that distinguishes the two
implementations -- a conforming allocator that is not a template, so its rebind
cannot be spelled by substituting a template argument. The hand-rolled code
returned such an allocator unchanged, which yields a vector<string> holding an
allocator that allocates int.

Two defects turned up while checking that the new documentation is true:

  * struct_view.hpp was not self-contained. The kIntrusiveIntrospection branch
    of PartialTie uses traits::kRowCategory and traits::RowCategoryType without
    including io/row_types.hpp, so the header compiled only because every
    translation unit reached it through row.hpp first. Including it on its own
    failed.

  * StructArgs did not compile at all: the closing angle bracket sat in the
    wrong place, handing Members... to remove_cvref_t and leaving StructView <
    to parse as a comparison. Nothing instantiated it, so nothing reported it.
    It is MakeStructView now -- the convention MakeKeysView, MakePortal and
    MakeQuery already follow, and a name that says what it returns -- and the
    write test instantiates it with a reordered pack, so it cannot regress
    unnoticed and the pack order is proven a second time.

The @file line also named partial_struct.hpp, which does not exist.

Documentation: reading a row through a view goes on the typed-results page,
beside the tuple, aggregate and Introspect cases it belongs with; passing a
structure as a statement's parameters goes in the execution section of the
running-queries page. Each page refers to the other rather than repeating it.
Both warnings there are measured rather than assumed -- in particular a view
cannot be combined with further arguments in one call, and the failure reads
only "Type doesn't have mapping to Postgres type." without naming the view.

Meta.* 15/15 and *StructView* 8/8 against a local postgres.
A container of structures can now be sent column by column through a view:
each element stays the structure, and the view picks the members that become
the statement's array parameters, in the order it names them.

    trx.ExecuteDecompose(query, io::StructViewContainer<NoEmailView>(users));
    trx.ExecuteDecomposeBulk(query, io::MakeViewContainer<&User::id, &User::name>(users));
    cluster.ExecuteDecompose(kMaster, query, io::MakeViewContainer<>(users));

StructViewContainer<View>(container) and MakeViewContainer<Members...>(container)
are helper functions returning detail::ViewContainer: C++20 class template
argument deduction cannot take the view explicitly and deduce the container.
MakeViewContainer deduces the structure from Container::value_type.

The column helpers in array_types.hpp shared two copies of a pfr-only
projection. They now share one ColumnProjection that goes through
View::Tie, with StructView<value_type> as the default view, and
DecomposeContainerByColumns / SplitContainerByColumns unwrap a ViewContainer
into its container and view. Transaction and Cluster are unchanged. As a side
effect, decomposing a container of structures with an intrusive Introspect()
works now; it was pfr aggregates only.

ExecuteBulk sends a chunk as a single array of the element type, and a subset
of a structure's members has no PostgreSQL type to map to, so passing a view
container there fails to compile with a hint to use ExecuteDecomposeBulk.
Cluster gets no bulk variant: bulk runs several statements and needs a
transaction around them.

AsContainer(RowTag) returned auto, so naming it forced its body to be
instantiated to deduce the type -- including inside a discarded if constexpr
branch of a non-template lambda, where the row-type static asserts then fired.
That is why a branch of ByteaWrapperRowType had been commented out. The result
types are declared again, through traits::RowExtractionT and
traits::ContainerRowExtractionT, Row::As(FieldTag) returns T, and the test
branch is restored.

StructView::Tie was constrained on is_convertible_v<U, UnderlyingType>, which
rejected non-copyable structures and accepted anything converting to the
structure, slicing it. It now requires a structure derived from the viewed one,
ties it as the viewed type, and takes an lvalue, so it cannot return references
into a temporary.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant