Unable to update K8s manifest file by sed

sysmicuser · 02-07-2021, 10:00 PM

I have the below YAML file as like below:

Code:

apiVersion: v1
items:
- apiVersion: v1
  kind: ServiceAccount
  metadata:
    creationTimestamp: 2019-03-20T23:17:42Z
    name: default
    namespace: dev4
    resourceVersion: "80999"
    selfLink: /api/v1/namespaces/dev4/serviceaccounts/default
    uid: 5c6e0d09-4b66-11e9-b4e3-0a779a87bb40
  secrets:
  - name: default-token-tl4dd
- apiVersion: v1
  kind: ServiceAccount
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"v1","kind":"ServiceAccount","metadata":{"annotations":{},"name":"pod-labeler","namespace":"dev4"}}
    creationTimestamp: 2020-04-21T05:46:25Z
    name: pod-labeler
    namespace: dev4
    resourceVersion: "113455688"
    selfLink: /api/v1/namespaces/dev4/serviceaccounts/pod-labeler
    uid: 702dadda-8393-11ea-abd9-0a768ca51346
  secrets:
  - name: pod-labeler-token-6vgp7
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

I am using sed to upate the file to be applied to a cluster, my sed command is like below:

Code:

sed -i \'/uid: \\|selfLink: \\|resourceVersion: \\|creationTimestamp: /d\' test1-serviceaccounts.yaml

However, I get the YAML file as like below:

Code:

apiVersion: v1
items:
- apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: default
    namespace: dev4
  secrets:
  - name: default-token-tl4dd
- apiVersion: v1
  kind: ServiceAccount
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"v1","kind":"ServiceAccount","metadata":{"annotations":{},"name":"pod-labeler","namespace":"dev4"}}
    name: pod-labeler
    namespace: dev4
  secrets:
  - name: pod-labeler-token-6vgp7
kind: List
metadata:

How should I modify my sed expression to get rid of the last line `metadata:`

Note: It should delete the last line.

I did try the below sed command however, it produces an error:

Updated sed command:

Code:

    sed -i '/uid: \|selfLink: \|resourceVersion: \|creationTimestamp: /$ d' 128-res-test-black-dev4-serviceaccounts.yaml

Error from executing the above command:

Code:

sed: -e expression #1, char 60: unknown command: `$'

The target file should look like this:

Code:

apiVersion: v1
items:
- apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: default
    namespace: dev4
  secrets:
  - name: default-token-tl4dd
- apiVersion: v1
  kind: ServiceAccount
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"v1","kind":"ServiceAccount","metadata":{"annotations":{},"name":"pod-labeler","namespace":"dev4"}}
    name: pod-labeler
    namespace: dev4
  secrets:
  - name: pod-labeler-token-6vgp7
kind: List

shruggy · 02-08-2021, 04:13 AM

First, there are tools that are much better than sed suited for manipulating the structured text data like YAML. E.g. see this list.

Among others, there are two different, but similar tools, both named yq, which are essentially the same to YAML what jq is to JSON: yq written in Python by Andrey Kislyuk, and yq written in Go by Mike Farah.

This is how I would do it with the Python yq:

Code:

yq -Yi \
 'del(.metadata)|
  del(..|.uid?,.selfLink?,.resourceVersion?,.creationTimestamp?)' \
  128-res-test-black-dev4-serviceaccounts.yaml

While the Python yq is just a wrapper around jq, the Go yq is a rewrite that currently supports only a subset of jq commands. So for it, the expression above should be rewritten slightly more verbose:

Code:

yq -i eval \
 'del(.metadata)|
  del(.items[].metadata|(.uid,.selfLink,.resourceVersion,.creationTimestamp))' \
  128-res-test-black-dev4-serviceaccounts.yaml

Quote:

Originally Posted by sysmicuser

Code:

sed -i '/uid: \|selfLink: \|resourceVersion: \|creationTimestamp: /$ d' 128-res-test-black-dev4-serviceaccounts.yaml

You cannot combine regex with numeric addresses for one command in sed (unless it's a range). Rewrite it using two d commands:

Code:

sed -i '/uid: \|selfLink: \|resourceVersion: \|creationTimestamp: /d;$d' 128-res-test-black-dev4-serviceaccounts.yaml

And using extended regular expressions this could be written more compactly:

Code:

sed -ri '/(uid|selfLink|resourceVersion|creationTimestamp): /d;$d' 128-res-test-black-dev4-serviceaccounts.yaml

sysmicuser · 02-12-2021, 01:26 AM

Thank you Shruggy