suggestion: add a default include_dir configuration

[mathroc]

Hi,

I find it harder than it should be to customize postgresql.conf:

• usually PGDATA is bind mounted so it’s not always easy to add a file in there at the right time (or replace a file)
• there’s options with /docker-entrypoint-initdb.d scripts but those are executed only when the database does not exists

so I suggest to add an include_dir configuration in the default postgresql.conf

in the Dockerfile, where listen_addresses is added, something along mkdir /docker-postgres-conf.d/ && echo 'include_dir=/docker-postgres-conf.d/' >> /usr/share/postgresql/postgresql.conf.sample

I believe this would be really useful, comes with no BC break and low maintenance

what do you think ?

(might be related to #581)

[marko-asplund]

Sometimes it can be necessary to be able to customize server config within the image instead of overriding via command line arguments to docker run, so +1 for this.

What do you think @wglambert ?

[marko-asplund]

Actually, would also be great if it would be possible to interpolate container start-up time environment variable values in config files. This would be useful e.g. in configuring cron.database_name parameter value for the pg_cron extension.

[yosifkit]

Using your suggestion, it is not that hard to extend the image for your own needs. Unfortunately, I don't think this is something we want to add to the image right now.

FROM postgres:13
RUN set eux; \
	confdir='/docker-postgres-conf.d/'; \
	mkdir -p "$confdir"; \
	chown postgres:postgres "$confdir"; \
	echo "include_dir = '$confdir'" >> /usr/share/postgresql/postgresql.conf.sample

$ docker build -t pg .
Sending build context to Docker daemon  11.26kB
Step 1/2 : FROM postgres:13
 ---> 293e4ed402ba
Step 2/2 : RUN set eux; 	confdir='/docker-postgres-conf.d/'; 	mkdir "$confdir"; 	chown postgres:postgres "$confdir"; 	echo "include_dir = '$confdir'" >> /usr/share/postgresql/postgresql.conf.sample
 ---> Running in 01feb0757294
Removing intermediate container 01feb0757294
 ---> c9083ff81afd
Successfully built c9083ff81afd
Successfully tagged pg:latest
$ cat extra.conf
shared_buffers=256MB
$ docker run -it --rm -e POSTGRES_PASSWORD=12345 --name pg-test -v "$PWD/extra.conf":/docker-postgres-conf.d/extra.conf pg
$ # check via another terminal and shared_buffers is correctly changed from the default

Conf files mounted this way or otherwise placed in the configuration directory would need to have proper permissions for the postgres user to read it.

[mathroc]

thx @yosifkit ! It’s indeed not too hard. Especially if we’re already build a custom image (eg: to install 3rd party extensions)

Still, I don’t understand why it should not be included in the base image, is it to keep as close as possible to the default Postgres configuration ? Or is it not useful enough ?

If you have a bit of time to spare, I would be interested in reading your point of view on the subject

I’m just asking out of curiosity, feel free to ignore this and close the issue if it’s not useful to keep this issue around. I won’t be offended !

And if you have time to waste, here is the thought process that led me to open this issue/suggestion :

For most project the default Postgres image is enough, and I can make it work by changing the configuration with command line arguments, but it’s not the nicest when you want to read / edit or look at configuration changes over time (because it’s - in my case - included in the docker-compose.yml file which should not, imho, be the place where I configure Postgres. The alternative is, as you mentioned, to build a custom image, either adding an include_dir in the default configuration file and mounting the configuration as a volume (or docker config) or adding the whole configuration into the image. Either way, it requires building and tagging a new image and probably pushing it to a registry. It’s not a huge task, but it’s tedious, and if you don’t already build custom images as part of your process it’s even more intimidating. I’m not a maintainer of widely use docker images, so I can definitely have missed something, but I thought this change comes with very little costs.

[thieltec]

postgresql.conf has the option of include files.
https://www.postgresql.org/docs/current/config-setting.html#CONFIG-INCLUDES

If this is always activated in the base postgresql.conf, it will be easier to add a customized configuration.

[oshmyrko]

They don't want to add one line into configuration file to simplify image usage for hundreds of people, people won't use their image. I personally prefer to use mysql, and this case is one of the reasons why.

[arist0v]

i would really like to have an option to specify an include_dirs,

to achieve it i:

• created a custom entrypoint script
• called the default one
• wait a little time then stop the service
• edit the .conf to add the include dir
• restart the service

#!/bin/bash
set -e

# Call the original entrypoint script to set up the environment
docker-entrypoint.sh postgres &

# Wait for the PostgreSQL server to be initialized and the postgresql.conf file to be generated
sleep 10

# Modify the postgresql.conf file
echo "include_dir = '/etc/my_folder/pgconf/'" >> /var/lib/postgresql/data/postgresql.conf

# Stop the PostgreSQL server that was started by the original entrypoint script
pg_ctl -D "$PGDATA" -m fast -w stop

# Now start the PostgreSQL server again with the modified configuration
exec postgres

it shouldn't be too hard to set in the entrypoint a verification if an include_dirs env var is set, and if path exist enable the include_dirs conf with this path in the original .conf before starting it, base use will still be able to use the current image as it, and advanced use will be able to use it as a base image and include custom .conf to be added

[mmomjian]

Has the team considered adding include_if_exists to the default postgresql.conf, which would not require the creation of an empty folder?

[bpow]

I'm just writing to add to the small chorus of people who would like this feature. I'm exploring moving from the bitnami images (which had the capacity for supplemental config files as described in this request), and was disappointed to see it absent in the main image without the workaround of making a derived docker image.

I think that a number of use cases for this functionality have been described over the course of several different issues/pull requests, primarily driven by the ideas of separating the main image itself from configuration. To me fits more to trying to keep the primary image as stateless as possible-- just being able to pull the image directly from the container repo.

On the other hand, from the primary dev team, I see messages saying "Unfortunately, I don't think this is something we want to add to the image right now." and in other related issues (e.g., #1267 saying this is the consensus among the dev group, but without stated rationale.

Like @kohlerdominik, I'm sensitive to the needs to keep things simple/managed for the maintainers of a large software project, but also think it would be helpful to have additional clarity on why this is their consensus to see of their concerns could be worked around in other ways (e.g., are there security concerns, anticipated differences between debian/alpine versions, or just trying to keep postgresql.conf as similar as possible to upstream?).

[tianon]

When we directly maintain an image on behalf of an upstream, especially when they aren't involved, we try to stick to their recommendations/preferences/defaults as much as possible, so that we're faithfully representing an "upstream" experience.

I believe that the example/pattern in #835 (comment) should still work.