Exported CSV Files are not UTF-8 Encoded

1

Hello,

When I download a .csv file, the character aren't well encoded.
For example, Priorité becomes Priorité
or Paramètrage becomes Paramétrage
I saw this issue on github https://github.com/metabase/metabase/issues/1961 but the issue still exist.
When I download using .xlsx, the character are well encoded.

{
  "browser-info": {
    "language": "fr-FR",
    "platform": "Win32",
    "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36 OPR/87.0.4390.35",
    "vendor": "Google Inc."
  },
  "system-info": {
    "file.encoding": "Cp1252",
    "java.runtime.name": "Java(TM) SE Runtime Environment",
    "java.runtime.version": "17.0.3.1+2-LTS-6",
    "java.vendor": "Oracle Corporation",
    "java.vendor.url": "https://java.oracle.com/",
    "java.version": "17.0.3.1",
    "java.vm.name": "Java HotSpot(TM) 64-Bit Server VM",
    "java.vm.version": "17.0.3.1+2-LTS-6",
    "os.name": "Windows Server 2016",
    "os.version": "10.0",
    "user.language": "fr",
    "user.timezone": "Europe/Paris"
  },
  "metabase-info": {
    "databases": [
      "postgres"
    ],
    "hosting-env": "unknown",
    "application-database": "mysql",
    "application-database-details": {
      "database": {
        "name": "MariaDB",
        "version": "10.6.5-MariaDB"
      },
      "jdbc-driver": {
        "name": "MariaDB Connector/J",
        "version": "2.7.5"
      }
    },
    "run-mode": "prod",
    "version": {
      "date": "2022-05-16",
      "tag": "v0.43.1",
      "branch": "release-x.43.x",
      "hash": "7f1a1c4"
    },
    "settings": {
      "report-timezone": "Europe/Paris"
    }
  }
}
2

Hi @Exe
Upgrade to latest release: https://github.com/metabase/metabase/releases/latest
Change your Java encoding by starting with -Dfile.encoding=UTF-8

I cannot reproduce, CSV are UTF-8 encoded. Try running this and exporting:

select 'Æbleflæsk' "Danish", 'Danish' "Æbleflæsk", 'παράδειγμα' "Greek", 'Greek' "παράδειγμα"
3

Using block note :
Danish,Æbleflæsk,Greek,παράδειγμαÆbleflæsk,Danish,παράδειγμα,Greek

Unsing excel software :
Danish,Æbleflæsk,Greek,παράδειγμαÆbleflæsk,Danish,παράδειγμα,Greek

4

@Exe Yes, because Excel is a terrible program that doesn't understand how standards work and requires BOM: https://github.com/metabase/metabase/issues/3271 - upvote by clicking :+1: on the first post

5

I did an upgrade to the latest version but I'm still getting the same issue on excel :frowning:
Is there any workaround ?

{
  "browser-info": {
    "language": "fr-FR",
    "platform": "Win32",
    "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36 OPR/87.0.4390.35",
    "vendor": "Google Inc."
  },
  "system-info": {
    "file.encoding": "Cp1252",
    "java.runtime.name": "Java(TM) SE Runtime Environment",
    "java.runtime.version": "17.0.3.1+2-LTS-6",
    "java.vendor": "Oracle Corporation",
    "java.vendor.url": "https://java.oracle.com/",
    "java.version": "17.0.3.1",
    "java.vm.name": "Java HotSpot(TM) 64-Bit Server VM",
    "java.vm.version": "17.0.3.1+2-LTS-6",
    "os.name": "Windows Server 2016",
    "os.version": "10.0",
    "user.language": "fr",
    "user.timezone": "Europe/Paris"
  },
  "metabase-info": {
    "databases": [
      "postgres"
    ],
    "hosting-env": "unknown",
    "application-database": "mysql",
    "application-database-details": {
      "database": {
        "name": "MariaDB",
        "version": "10.6.5-MariaDB"
      },
      "jdbc-driver": {
        "name": "MariaDB Connector/J",
        "version": "2.7.5"
      }
    },
    "run-mode": "prod",
    "version": {
      "date": "2022-06-13",
      "tag": "v0.43.3",
      "branch": "release-x.43.x",
      "hash": "c9c7ef0"
    },
    "settings": {
      "report-timezone": "Europe/Paris"
    }
  }
}
6

@Exe Yes, don't use CSV in Excel, use XLSX.

7

How can I add this setting to java ?

8

@Exe java -Dfile.encoding=UTF-8 ...other settings... -jar metabase.jar

9

I'm running Metabase using nssm

10

@Exe And you'll add it to the Arguments before the -jar metabase.jar

It won't fix anything. You will continue to see problems in Excel, since Excel requires BOM

1 Like