Scripted fields incompatible with Elasticsearch/Kibana v8

[fholzer]

We're on version 7.x and the upgrade assistant gives the following output:

You have 1 index patterns (cf-http-*...) that use scripted fields. Scripted fields are deprecated and will be removed in future. Use runtime fields instead.

Issues reported by the upgrade assistant must fixed before upgrading to v8. Seems like multiple fields are scripted. That logic will likely need to be moved from scripted fields to either runtime fields or the ingest pipeline.

Reference:
https://www.elastic.co/de/blog/introducing-elasticsearch-runtime-fields
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/runtime.html

[ejcx]

Hi, this project is unfortunately not very active and we haven't accommodated for updates in elasticsearch or kibana. I'm not the maintainer of this project but if you have a pull request that addresses the issue I'd be more than happy to look at it.

[fholzer]

Memo to myself:
To extract scripted fields from exported kibana saved-object files, changing their data structure to the one used for dynamic fields:

jq '.[] | select(._type == "index-pattern")._source.fields | fromjson | reduce (.[] | select(.scripted == true)) as $i ({}; . += { ($i.name): { type: $i.type, script: { source: $i.script } } })' dashboards/dashboards.json

Note that the script field content still needs to be updated to use emit() as per docs. That's already done in #37

[penekk]

I have managed to "migrate" this to runtime fields within index-template some time ago that I plan to start using soon - I decided to use https://github.com/elastic/terraform-provider-elasticstack for that. Once I find some time I could share those snippets, if you like. I do remember there were some challenges with reformatting everything so that you can push the json properly but still maintain some readability of .tf files.