Pandoc 缺失导致 SparkR 构建失败
# 问题现象
在 Spark 3.5.5 通过 Bigtop 体系编译 RPM 包的过程中,执行到 SparkR 相关模块时,构建日志持续输出如下异常:
++ [[ -f /opt/modules/bigtop/build/spark/rpm/BUILD/spark-3.5.5/conf/spark-env.sh ]]
++ '[' -z '' ']'
++ SCALA_VERSION_1=2.13
++ SCALA_VERSION_2=2.12
++ ASSEMBLY_DIR_1=/opt/modules/bigtop/build/spark/rpm/BUILD/spark-3.5.5/assembly/target/scala-2.13
++ ASSEMBLY_DIR_2=/opt/modules/bigtop/build/spark/rpm/BUILD/spark-3.5.5/assembly/target/scala-2.12
++ ENV_VARIABLE_DOC=https://spark.apache.org/docs/latest/configuration.html#environment-variables
++ [[ -d /opt/modules/bigtop/build/spark/rpm/BUILD/spark-3.5.5/assembly/target/scala-2.13 ]]
++ [[ -d /opt/modules/bigtop/build/spark/rpm/BUILD/spark-3.5.5/assembly/target/scala-2.13 ]]
++ export SPARK_SCALA_VERSION=2.12
++ SPARK_SCALA_VERSION=2.12
+++ ps -o stat= -p 81435
++ [[ ! S =~ \+ ]]
++ [[ -p /dev/stdin ]]
+ '[' -f /opt/modules/bigtop/build/spark/rpm/BUILD/spark-3.5.5/RELEASE ']'
+ SPARK_JARS_DIR=/opt/modules/bigtop/build/spark/rpm/BUILD/spark-3.5.5/assembly/target/scala-2.12/jars
+ '[' -d /opt/modules/bigtop/build/spark/rpm/BUILD/spark-3.5.5/assembly/target/scala-2.12/jars ']'
+ SPARK_HOME=/opt/modules/bigtop/build/spark/rpm/BUILD/spark-3.5.5
+ /usr/local/R-4.4.2/bin/R CMD build /opt/modules/bigtop/build/spark/rpm/BUILD/spark-3.5.5/R/pkg
WARNING: ignoring environment value of R_HOME
* checking for file '/opt/modules/bigtop/build/spark/rpm/BUILD/spark-3.5.5/R/pkg/DESCRIPTION' ... OK
* preparing 'SparkR':
* checking DESCRIPTION meta-information ... OK
* installing the package to build vignettes
* creating vignettes ... ERROR[33m 25s]
--- re-building 'sparkr-vignettes.Rmd' using rmarkdown
Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics:
Pandoc is required to build R Markdown vignettes but not available. Please make sure it is installed.
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
--- failed re-building 'sparkr-vignettes.Rmd'
SUMMARY: processing the following file failed:
'sparkr-vignettes.Rmd'
Error: Vignette re-building failed.
Execution halted
error: Bad exit status from /var/tmp/rpm-tmp.JSj4Te (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.JSj4Te (%build)
> Task :spark-rpm FAILED
FAILURE: Build failed with an exception.
* Where:
Script '/opt/modules/bigtop/packages.gradle' line: 545
* What went wrong:
Execution failed for task ':spark-rpm'.
> Process 'command 'rpmbuild'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with
--scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 33m 26s
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
报错关键点
从日志中可以明确看到:
Pandoc is required to build R Markdown vignettes but not available.
Error in eval(x, envir = envir) : object 'EVAL_CHUNK' not found
说明 Pandoc 未安装 是根因,缺失 Pandoc 会导致 SparkR 包构建中的 vignette 生成环节全部失败。
# 原因分析
SparkR 的构建流程默认依赖 R Markdown(rmarkdown 包),而 vignettes 的渲染实际需要调用 Pandoc。如果系统未装 Pandoc,则无论是本地 R CMD build 还是集成到 RPM 过程都会失败。此类环境依赖在 R 社区编译型组件常见,特别是涉及文档自动化和测试用例生成时。
笔记
即使提前装好 R、rmarkdown、knitr 等 R 包,也无法绕开 Pandoc 的二进制依赖!
# 解决方案
只需在目标环境补全 Pandoc 即可。
dnf install -y pandoc
1
若你是 CentOS 7/8 或 RHEL 用户,也可通过 yum install pandoc
或结合 epel-release 安装。
提示
安装完成后,建议重新启动 Spark 构建流程,确保 R CMD build 能成功通过,Vignette 能被正确生成。