Presto自定義函數(shù)@SqlNullable引發(fā)問題詳解

更新時間：2022年12月05日 11:39:21 作者：EdurtIO

這篇文章主要為大家介紹了Presto自定義函數(shù)@SqlNullable引發(fā)問題詳解，有需要的朋友可以借鑒參考下，希望能夠有所幫助，祝大家多多進步，早日升職加薪

引發(fā)Presto問題

看到標題我們會想到是由于@SqlNullable注解引發(fā)的問題，我們先看一段代碼，正是這段有意思的代碼，讓我糾結了2個多小時，引發(fā)了Presto的問題。

@Description("user_id")
@ScalarFunction("user_id")
@SqlType(StandardTypes.VARCHAR)
public static Slice userId(@SqlType(StandardTypes.VARCHAR) Slice value) {
    String _value = value.toStringUtf8();
    if (StringUtils.containsWhitespace(_value)) {
        _value = StringUtils.replace(_value, " ", "+");
    } 
    return Slices.utf8Slice(makeErrorMsgBase64(_value));
}

這段代碼很簡單，就是我們將傳遞進來的base64的字符串解碼成實際的字符串，單單從代碼上看是不會有什么問題的。當我們實際運行這個函數(shù)的時候問題就出現(xiàn)了，以下是一個使用示例：

select user_id(str) from temp.users limit 100;

看執(zhí)行的SQL似乎也沒有什么異常，但就是這么簡簡單單的一個函數(shù)引發(fā)了Presto的問題，那就是java.lang.NullPointerException: undefined錯誤

錯誤具體內容

java.lang.NullPointerException: undefined
	at io.prestosql.type.VarcharOperators.equal(VarcharOperators.java:53)
	at io.prestosql.$gen.CursorProcessor_20200927_063218_2398.filter(Unknown Source)
	at io.prestosql.$gen.CursorProcessor_20200927_063218_2398.process(Unknown Source)
	at io.prestosql.operator.ScanFilterAndProjectOperator$RecordCursorToPages.process(ScanFilterAndProjectOperator.java:323)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
	at io.prestosql.operator.WorkProcessorUtils$YieldingProcess.process(WorkProcessorUtils.java:181)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
	at io.prestosql.operator.WorkProcessorUtils.lambda$processStateMonitor$2(WorkProcessorUtils.java:200)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.lambda$flatten$6(WorkProcessorUtils.java:277)
	at io.prestosql.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:319)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:306)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
	at io.prestosql.operator.WorkProcessorUtils.lambda$processStateMonitor$2(WorkProcessorUtils.java:200)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
	at io.prestosql.operator.WorkProcessorUtils.lambda$finishWhen$3(WorkProcessorUtils.java:215)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorSourceOperatorAdapter.getOutput(WorkProcessorSourceOperatorAdapter.java:149)
	at io.prestosql.operator.Driver.processInternal(Driver.java:379)
	at io.prestosql.operator.Driver.lambda$processFor$8(Driver.java:283)
	at io.prestosql.operator.Driver.tryWithLock(Driver.java:675)
	at io.prestosql.operator.Driver.processFor(Driver.java:276)
	at io.prestosql.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1076)
	at io.prestosql.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:shichengoooo@163.com)
	at io.prestosql.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:484)
	at io.prestosql.$gen.Presto_341____20200925_110330_2.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

修改UDF源碼

通過上面錯誤我們推算出應該是由于數(shù)據(jù)導致空指針異常，那么問題出在哪里呢？問題就出在查詢str這個字段中，這個字段我們經過實際的查詢發(fā)現(xiàn)是有''的數(shù)據(jù)，在做轉換的時候出現(xiàn)了空指針問題，于是我們修改UDF的源碼為

@Description("user_id")
@ScalarFunction("user_id")
@SqlType(StandardTypes.VARCHAR)
@SqlNullable
public static Slice userId(@SqlNullable @SqlType(StandardTypes.VARCHAR) Slice value) {
    String _value = value.toStringUtf8();
    if (StringUtils.containsWhitespace(_value)) {
        _value = StringUtils.replace(_value, " ", "+");
    } 
    return Slices.utf8Slice(makeErrorMsgBase64(_value));
}

我們在方法和參數(shù)上添加了@SqlNullable注解用于標記此函數(shù)可以接收空的數(shù)據(jù)，這似乎看著也沒有問題，我們將該函數(shù)重新發(fā)布，再次執(zhí)行SQL發(fā)現(xiàn)還存在相同的問題，于是又將代碼修改為以下內容

@Description("user_id")
@ScalarFunction("user_id")
@SqlType(StandardTypes.VARCHAR)
public static Slice userId(@SqlNullable @SqlType(StandardTypes.VARCHAR) Slice value) {
    String _value = value.toStringUtf8();
    if (StringUtils.containsWhitespace(_value)) {
        _value = StringUtils.replace(_value, " ", "+");
    } 
    return Slices.utf8Slice(makeErrorMsgBase64(_value));
}

刪除了方法上的@SqlNullable注解，再次運行發(fā)現(xiàn)不會再出現(xiàn)這個錯誤，但是Presto服務中不斷的報出空指針錯誤，只是不在反饋給查詢客戶端，原本以為此問題已經解決，然而更有意思的事情發(fā)生了，我們使用了343版本測試成功后，線上版本是341，升級線上后發(fā)現(xiàn)此問題再次復現(xiàn)，如果再次在方法上加上@SqlNullable注解在341版本上又會修復這個問題，目前已將這個問題反饋給官方，推薦大家使用343版本！

以上就是Presto自定義函數(shù)@SqlNullable引發(fā)問題詳解的詳細內容，更多關于Presto自定義函數(shù)@SqlNullable的資料請關注腳本之家其它相關文章！

您可能感興趣的文章: