Visual editor generates incorrect SQL

Hello,
I guess I found case when visual editor generates incorrect SQL.


Generates the following SQL statement:
SELECT
*
FROM
(
SELECT
"SOA"."REL_INC_PROBS_MV"."SOURCE" AS "SOURCE",
"SOA"."REL_INC_PROBS_MV"."DEPEND" AS "DEPEND",
"SOA"."REL_INC_PROBS_MV"."TYPE" AS "TYPE",
"Группа Обращений>Обращения - Код обращения"."ID" AS "identifier_nrzurztxqzz",
"Группа Обращений>Обращения - Код обращения"."INCIDENT_ID" AS "identifier_nsuvruttzz",
"Группа Обращений>Обращения - Код обращения"."ADDED_BY" AS "identifier_nrxyvyvwwzz",
"Группа Обращений>Обращения - Код обращения"."ADDED_TIME" AS "identifier_uuwzvtvvx",
"Группа Обращений>Обращения - Код обращения"."REASON" AS "identifier_nrqvzvtvvrw",
"Группы обращений - ID"."AFFECTED_ITEM" AS "identifier_rqsxrtrvsq",
"Группы обращений - ID"."ASSIGNEE" AS "identifier_nxxwuzxqrt",
"Группы обращений - ID"."ASSIGNMENT" AS "identifier_nrwwrzvwwxt",
"Группы обращений - ID"."CATEGORY" AS "identifier_nruyruqyvxu",
"Группы обращений - ID"."CLOSE_TIME" AS "identifier_rvxxsvvsqx",
"Группы обращений - ID"."DESCRIPTION" AS "identifier_rxqwxtxtxx",
"Группы обращений - ID"."DIRECTION" AS "identifier_nrquvrvvtvv",
"Группы обращений - ID"."FZ_NUMBER" AS "identifier_nwsvzqwusv",
"Группы обращений - ID"."GROUPSERVICE" AS "identifier_rsxqzstxyt",
"Группы обращений - ID"."ID" AS "Группы обращений - ID__ID",
"Группы обращений - ID"."INCIDENTS_COUNT" AS "identifier_nuuvqwyuss",
"Группы обращений - ID"."OPEN_TIME" AS "identifier_rsrzttwyxr",
"Группы обращений - ID"."SERVICE" AS "Группы обращений - ID__SERVICE",
"Группы обращений - ID"."SUBCATEGORY" AS "identifier_nvuqszzzvv",
"Группы обращений - ID"."SYSMODTIME" AS "identifier_nrttwztyzsv",
"Группы обращений - ID"."TITLE" AS "Группы обращений - ID__TITLE"
FROM
"SOA"."REL_INC_PROBS_MV"
LEFT JOIN "SOA"."INCIDENT_GROUP_MEMBERS" "identifier_ztwtrywzy" ON "SOA"."REL_INC_PROBS_MV"."SOURCE" = "Группа Обращений>Обращения - Код обращения"."INCIDENT_ID"
LEFT JOIN "SOA"."INCIDENT_GROUPS" "Группы обращений - ID" ON "Группа Обращений>Обращения - Код обращения"."ID" = "Группы обращений - ID"."ID"
)
WHERE
ROWNUM <= 1048575
Oracle response is:
ORA-00904: "Группа Обращений>Обращения - Код обращения"."INCIDENT_ID": invalid identifier
As far as I understand "Группа Обращений>Обращения - Код обращения"."INCIDENT_ID" was renamed to "identifier_nsuvruttzz", but in join expression its referenced by original name.
First time the error was met in v 0.38.2. After upgrade to v0.39.1 still unchanged. Source database Oracle 12.1.
Diagnostic info
{
"browser-info": {
"language": "en-gb",
"platform": "MacIntel",
"userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15",
"vendor": "Apple Computer, Inc."
},
"system-info": {
"file.encoding": "UTF-8",
"java.runtime.name": "Java(TM) SE Runtime Environment",
"java.runtime.version": "1.8.0_261-b12",
"java.vendor": "Oracle Corporation",
"java.vendor.url": "http://java.oracle.com/",
"java.version": "1.8.0_261",
"java.vm.name": "Java HotSpot(TM) 64-Bit Server VM",
"java.vm.version": "25.261-b12",
"os.name": "Windows Server 2019",
"os.version": "10.0",
"user.language": "en",
"user.timezone": "Europe/Moscow"
},
"metabase-info": {
"databases": [
"oracle"
],
"hosting-env": "unknown",
"application-database": "postgres",
"application-database-details": {
"database": {
"name": "PostgreSQL",
"version": "12.4"
},
"jdbc-driver": {
"name": "PostgreSQL JDBC Driver",
"version": "42.2.18"
}
},
"run-mode": "prod",
"version": {
"tag": "v0.39.0",
"date": "2021-05-04",
"branch": "master",
"hash": "e0c624b"
},
"settings": {
"report-timezone": "Europe/Moscow"
}
}
}

Hi @Max.A
Nothing gets fixed automagically unless it has been reported.
Please test with an official release.
Which version of ojdbc8.jar are you using?
What is the setting of Admin > Settings > General > "Friendly table and field names" ?

Hi @flamber
I've got some result with official release, but only using Postgres db.
On my test site with H2 everything is Ok.
Oracle ojdbc8 19.3.0.0.0
"Firendly table and field" name is disabled
Diagnostic info
{
"browser-info": {
"language": "en-gb",
"platform": "MacIntel",
"userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15",
"vendor": "Apple Computer, Inc."
},
"system-info": {
"file.encoding": "UTF-8",
"java.runtime.name": "Java(TM) SE Runtime Environment",
"java.runtime.version": "1.8.0_261-b12",
"java.vendor": "Oracle Corporation",
"java.vendor.url": "http://java.oracle.com/",
"java.version": "1.8.0_261",
"java.vm.name": "Java HotSpot(TM) 64-Bit Server VM",
"java.vm.version": "25.261-b12",
"os.name": "Windows Server 2019",
"os.version": "10.0",
"user.language": "en",
"user.timezone": "Europe/Moscow"
},
"metabase-info": {
"databases": [
"oracle"
],
"hosting-env": "unknown",
"application-database": "postgres",
"application-database-details": {
"database": {
"name": "PostgreSQL",
"version": "12.4"
},
"jdbc-driver": {
"name": "PostgreSQL JDBC Driver",
"version": "42.2.18"
}
},
"run-mode": "prod",
"version": {
"tag": "v0.39.1",
"date": "2021-04-27",
"branch": "release-x.39.x",
"hash": "6beba48"
},
"settings": {
"report-timezone": "Europe/Moscow"
}
}
}
Editor


Result:
image
Generated query:
SELECT *
FROM (SELECT "SOA"."INCIDENT_GROUP_MEMBERS"."ID" AS "ID", "SOA"."INCIDENT_GROUP_MEMBERS"."INCIDENT_ID" AS "INCIDENT_ID", "SOA"."INCIDENT_GROUP_MEMBERS"."ADDED_BY" AS "ADDED_BY", "SOA"."INCIDENT_GROUP_MEMBERS"."ADDED_TIME" AS "ADDED_TIME", "SOA"."INCIDENT_GROUP_MEMBERS"."REASON" AS "REASON", "Обращения>Инциденты - Код обращения"."SOURCE" AS "identifier_uryqtzqqv", "Обращения>Инциденты - Код обращения"."DEPEND" AS "identifier_nrzrxzvszwz", "Обращения>Инциденты - Код обращения"."TYPE" AS "identifier_tyrrwxyzv" FROM "SOA"."INCIDENT_GROUP_MEMBERS"
LEFT JOIN "SOA"."REL_INC_PROBS_MV" "identifier_nwqzytvtqu" ON "SOA"."INCIDENT_GROUP_MEMBERS"."INCIDENT_ID" = "Обращения>Инциденты - Код обращения"."SOURCE")
WHERE rownum <= 1048575

@Max.A I don't understand "but only using Postgres db" - are you saying that you're having this problem with Postgres as data source too? How does the SQL look for Postgres?

No, I have two installations Product with Postgres as a metabase db, and test with h2. Both use same oracle as data source.
The test works good, only product has this error.

@Max.A Okay, so you have "testing" and "production" instances, and it only fails in "production"?
Then post "Diagnostic Info" for both instances.
And make sure that you are using the same Oracle driver dependencies (in the plugins directory) on both instances.

Hi @flamber
Yes, fails in production only.
Test and production instance are same, except metabase db and os version.
Production instance Windows 2019
Test instance Windows 2012R2

Test instance
Oracle ojdbc8 19.3.0.0.0
"Firendly table and field name" is disabled
Diagnostic Info
{
"browser-info": {
"language": "en-gb",
"platform": "MacIntel",
"userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15",
"vendor": "Apple Computer, Inc."
},
"system-info": {
"file.encoding": "Cp1252",
"java.runtime.name": "Java(TM) SE Runtime Environment",
"java.runtime.version": "1.8.0_162-b12",
"java.vendor": "Oracle Corporation",
"java.vendor.url": "http://java.oracle.com/",
"java.version": "1.8.0_162",
"java.vm.name": "Java HotSpot(TM) 64-Bit Server VM",
"java.vm.version": "25.162-b12",
"os.name": "Windows Server 2012 R2",
"os.version": "6.3",
"user.language": "en",
"user.timezone": "Europe/Moscow"
},
"metabase-info": {
"databases": [
"oracle",
"h2"
],
"hosting-env": "unknown",
"application-database": "h2",
"application-database-details": {
"database": {
"name": "H2",
"version": "1.4.197 (2018-03-18)"
},
"jdbc-driver": {
"name": "H2 JDBC Driver",
"version": "1.4.197 (2018-03-18)"
}
},
"run-mode": "prod",
"version": {
"tag": "v0.39.1",
"date": "2021-04-27",
"branch": "release-x.39.x",
"hash": "6beba48"
},
"settings": {
"report-timezone": null
}
}
}

Production instance
Oracle ojdbc8 19.3.0.0.0
"Firendly table and field name" is disabled
Diagnostic info
{
"browser-info": {
"language": "en-gb",
"platform": "MacIntel",
"userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15",
"vendor": "Apple Computer, Inc."
},
"system-info": {
"file.encoding": "UTF-8",
"java.runtime.name": "Java(TM) SE Runtime Environment",
"java.runtime.version": "1.8.0_261-b12",
"java.vendor": "Oracle Corporation",
"java.vendor.url": "http://java.oracle.com/",
"java.version": "1.8.0_261",
"java.vm.name": "Java HotSpot(TM) 64-Bit Server VM",
"java.vm.version": "25.261-b12",
"os.name": "Windows Server 2019",
"os.version": "10.0",
"user.language": "en",
"user.timezone": "Europe/Moscow"
},
"metabase-info": {
"databases": [
"oracle"
],
"hosting-env": "unknown",
"application-database": "postgres",
"application-database-details": {
"database": {
"name": "PostgreSQL",
"version": "12.4"
},
"jdbc-driver": {
"name": "PostgreSQL JDBC Driver",
"version": "42.2.18"
}
},
"run-mode": "prod",
"version": {
"tag": "v0.39.1",
"date": "2021-04-27",
"branch": "release-x.39.x",
"hash": "6beba48"
},
"settings": {
"report-timezone": "Europe/Moscow"
}
}
}

@Max.A Noticeable differences besides to OS is the Java version and file.encoding.
I would recommend that you try using the same on production.
You might also want to test with newer versions of Java and ojdbc8.jar, and you might need to use orai18n.jar too.
https://www.metabase.com/docs/latest/operations-guide/java-versions.html
https://www.oracle.com/database/technologies/appdev/jdbc-ucp-21-1-c-downloads.html

@flamber I made test instance closer to production one: file encoding=UTF-8 and Java version 1.8.0_261.
It doesn't make any effect.
Diagnostic info for test instance
{
"browser-info": {
"language": "en-gb",
"platform": "MacIntel",
"userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15",
"vendor": "Apple Computer, Inc."
},
"system-info": {
"file.encoding": "UTF-8",
"java.runtime.name": "Java(TM) SE Runtime Environment",
"java.runtime.version": "1.8.0_261-b12",
"java.vendor": "Oracle Corporation",
"java.vendor.url": "http://java.oracle.com/",
"java.version": "1.8.0_261",
"java.vm.name": "Java HotSpot(TM) 64-Bit Server VM",
"java.vm.version": "25.261-b12",
"os.name": "Windows Server 2012 R2",
"os.version": "6.3",
"user.language": "en",
"user.timezone": "Europe/Moscow"
},
"metabase-info": {
"databases": [
"oracle",
"h2"
],
"hosting-env": "unknown",
"application-database": "h2",
"application-database-details": {
"database": {
"name": "H2",
"version": "1.4.197 (2018-03-18)"
},
"jdbc-driver": {
"name": "H2 JDBC Driver",
"version": "1.4.197 (2018-03-18)"
}
},
"run-mode": "prod",
"version": {
"tag": "v0.39.1",
"date": "2021-04-27",
"branch": "release-x.39.x",
"hash": "6beba48"
},
"settings": {
"report-timezone": null
}
}
}

I guess its hardly ojdbs related problem - generated sql has error:
joined table JOIN "SOA"."REL_INC_PROBS_MV" alias is "identifier_nwqzytvtqu",
but in join condition the table referenced as "Обращения>Инциденты - Код обращения".
"Обращения>Инциденты - Код обращения" is a combination of name table "REL_INC_PROBS_MV" in Data model and column name of table "INCIDENT_GROUP_MEMBERS" used as key for join.

Incorrect query example
SELECT *
FROM (SELECT "SOA"."INCIDENT_GROUP_MEMBERS"."ID"
AS "ID",
"SOA"."INCIDENT_GROUP_MEMBERS"."INCIDENT_ID"
AS "INCIDENT_ID",
"SOA"."INCIDENT_GROUP_MEMBERS"."ADDED_BY"
AS "ADDED_BY",
"SOA"."INCIDENT_GROUP_MEMBERS"."ADDED_TIME"
AS "ADDED_TIME",
"SOA"."INCIDENT_GROUP_MEMBERS"."REASON"
AS "REASON",
"Обращения>Инциденты - Код обращения"."SOURCE"
AS "identifier_uryqtzqqv",
"Обращения>Инциденты - Код обращения"."DEPEND"
AS "identifier_nrzrxzvszwz",
"Обращения>Инциденты - Код обращения"."TYPE"
AS "identifier_tyrrwxyzv"
FROM "SOA"."INCIDENT_GROUP_MEMBERS"
LEFT JOIN "SOA"."REL_INC_PROBS_MV" "identifier_nwqzytvtqu"
ON "SOA"."INCIDENT_GROUP_MEMBERS"."INCIDENT_ID" =
"Обращения>Инциденты - Код обращения"."SOURCE")
WHERE ROWNUM <= 1048575

correct query generated by test instance
SELECT *
FROM (SELECT "SOA"."INCIDENT_GROUP_MEMBERS"."ID"
AS "ID",
"SOA"."INCIDENT_GROUP_MEMBERS"."INCIDENT_ID"
AS "INCIDENT_ID",
"SOA"."INCIDENT_GROUP_MEMBERS"."ADDED_BY"
AS "ADDED_BY",
"SOA"."INCIDENT_GROUP_MEMBERS"."ADDED_TIME"
AS "ADDED_TIME",
"SOA"."INCIDENT_GROUP_MEMBERS"."REASON"
AS "REASON",
"Обращения>Инциденты"."SOURCE"
AS "Обращения>Инциденты__SOURCE",
"Обращения>Инциденты"."DEPEND"
AS "Обращения>Инциденты__DEPEND",
"Обращения>Инциденты"."TYPE"
AS "Обращения>Инциденты__TYPE"
FROM "SOA"."INCIDENT_GROUP_MEMBERS"
LEFT JOIN
"SOA"."REL_INC_PROBS_MV"
"Обращения>Инциденты"
ON "SOA"."INCIDENT_GROUP_MEMBERS"."INCIDENT_ID" =
"Обращения>Инциденты"."SOURCE")
WHERE ROWNUM <= 1048575

@Max.A That took a while to figure out what was going on - I have created an issue about this:
https://github.com/metabase/metabase/issues/15978 - upvote by clicking :+1: on the first post

Hi @flamber
Unfortunately it’s not my case. I’ve tried to change field aliases.
By the way, the situation you described can be solved by using oracle 12.2 with parameter compatible=‘12.2.0’. It makes maximum variable name length equal to 128 bytes.
I suggest that something wrong with creating sql expression. I would like try to check it. Could you point to metabase module responsible for it?

@Max.A Yes, something is wrong with the aliasing, which is what the issue is about. It works if you have less than 30 bytes for the alias, but you have to remember that Metabase is aliasing by combining your table and column, which is likely what is causing you problems.

Changing Oracle to allow larger aliases won't fix the problem, since Metabase handles it wrong.

The identifier code is in the driver, but I guess there's a missing function there, which should also handle the join part.
https://github.com/metabase/metabase/blob/master/modules/drivers/oracle/src/metabase/driver/oracle.clj

Hi @flamber, you are right.
The problem is related to the maximum name length in metabase oracle driver.
I've changed legacy-max-identifier-length from 30 to 128.
It solved problem for oracle 12.2.