Skip to content

Windows - NULL byte around each character #2622

@reini-1

Description

@reini-1

Describe the bug
yq outputfile created on windows inclused lots of NULL bytes around each character:

Image

A clear and concise description of what the bug is.

I use yq to "explode" yaml anchors in a file because winget config can not handly anchors, but when I create this on Windows 11, the resulting file is somehow strange, with all these NULL bytes, see the screenshot from vs code.

I use the following command do create the exploded yaml file:

$ yq "explode(.)" --yaml-fix-merge-anchor-to-spec test.yml > test.winget.yml

Version of yq: 4.X.X
$ yq --version
yq (https://github.com/mikefarah/yq/) version v4.52.4

Operating system: mac/linux/windows/....
$ cmd /c ver
Microsoft Windows [Version 10.0.26200.7840]

Installed via: docker/binary release/homebrew/snap/...
$ winget install MikeFarah.yq

Input Yaml
Concise yaml document(s) (as simple as possible to show the bug, please keep it to 10 lines or less)

test.yml:
test.yml

---
# yaml-language-server: $schema=https://aka.ms/configuration-dsc-schema/0.2

properties:
  configurationVersion: 0.2.0
  .user_package: &user-package
    resource: Microsoft.WinGet.DSC/WinGetPackage
    settings:
      source: winget

  resources:
    - <<: *user-package
      settings:
        id: dandavison.delta

    - <<: *user-package
      settings:
        id: MikeFarah.yq

Command
The command you ran:

yq "explode(.)" --yaml-fix-merge-anchor-to-spec test.yml > test.winget.yml

Actual behaviour

test.winget.yml:
test.winget.yml

Expected behaviour

test.winget.yml

---
# yaml-language-server: $schema=https://aka.ms/configuration-dsc-schema/0.2

properties:
  configurationVersion: 0.2.0
  .user_package:
    resource: Microsoft.WinGet.DSC/WinGetPackage
    settings:
      source: winget
  resources:
    - resource: Microsoft.WinGet.DSC/WinGetPackage
      settings:
        id: dandavison.delta
    - resource: Microsoft.WinGet.DSC/WinGetPackage
      settings:
        id: MikeFarah.yq

Additional context
When I run this from WSL accessing the file on the Windows side via /mnt/c/User/... and also writing to the Windows side it works. So I think it is a problem of the windows version.

$ echo $WSL_DISTRO_NAME; command -v yq; yq --version
Ubuntu-24.04
/home/reini/bin/yq
yq (https://github.com/mikefarah/yq/) version v4.52.4

running the native windows command from within msys32 it also works:

$ C:/Users/reini/AppData/Local/Microsoft/WinGet/Packages/MikeFarah.yq_Microsoft.Winget.Source_8wekyb3d8bbwe/yq.exe "explode(.)" --yaml-fix-merge-anchor-to-spec test.yml > test.winget.yml

Only running it directly in Windows system powershell did not work.

$ $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.26100.7705
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.26100.7705
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Using Powershell 7 it works:

$ $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.5.4
PSEdition                      Core
GitCommitId                    7.5.4
OS                             Microsoft Windows 10.0.26200
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Calling it via cmd.exe from system powershell also creates a lot of NULL:

$ cmd /c yq "explode(.)" --yaml-fix-merge-anchor-to-spec test.yml > test.winget.yml

Calling it via cmd.exe the same way from powershell 7 works.

Directly open a cmd.exe shell also works.

So it seems to be a problem with the system powershell.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions