Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-8338] Ganglia fails to run because of httpd.conf configuration mismatch #121

Closed
wants to merge 7 commits into from
Closed

[SPARK-8338] Ganglia fails to run because of httpd.conf configuration mismatch #121

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
216bb02
update ganglia, httpd, php
Jun 13, 2015
5fd7d37
fix php version mismatch
Jun 13, 2015
fce4dad
fixed typo
Jun 13, 2015
f6c627a
wait till yum uninstall finished on server and slaves
Aug 4, 2015
6ba97bb
hide warning when packages were not installed before
Aug 4, 2015
cc95f0a
block till other yum process finishes its job
Aug 4, 2015
ed3f246
Revert "block till other yum process finishes its job"
Aug 4, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions ganglia/init.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,19 @@ rm -rf /mnt/ganglia/rrds/*
mkdir -p /mnt/ganglia/rrds
chown -R nobody:nobody /mnt/ganglia/rrds

# Install ganglia
# TODO: Remove this once the AMI has ganglia by default
OLD_GANGLIA_PACKAGES="httpd* php* ganglia* ganglia* ganglia-gmond* ganglia-gmetad*"
GANGLIA_PACKAGES="httpd24-2.4* php56-5.6* ganglia-3.6* ganglia-web-3.5* ganglia-gmond-3.6* ganglia-gmetad-3.6*"

GANGLIA_PACKAGES="ganglia ganglia-web ganglia-gmond ganglia-gmetad"
#Uninstalls older version of ganglia from master if it was reinstalled in AMI
yum remove -q -y $OLD_GANGLIA_PACKAGES & sleep 1
wait
yum install -q -y $GANGLIA_PACKAGES & sleep 1
wait

if ! rpm --quiet -q $GANGLIA_PACKAGES; then
yum install -q -y $GANGLIA_PACKAGES;
fi
for node in $SLAVES $OTHER_MASTERS; do
ssh -t -t $SSH_OPTS root@$node "if ! rpm --quiet -q $GANGLIA_PACKAGES; then yum install -q -y $GANGLIA_PACKAGES; fi" & sleep 0.3
#Uninstalls older version of ganglia from other masters if it was reinstalled in AMI
ssh -t -t $SSH_OPTS root@$node "yum remove -q -y $OLD_GANGLIA_PACKAGES" & sleep 0.3
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this out today and I ran into error messages of the form Existing lock /var/run/yum.pid: another copy is running as pid 2758.

I think the problem is that while the yum remove is running we are also starting the yum install. I think a good thing to do here would be to not put the ssh in background in the first line. Also we can probably use pssh to parallelize these steps rather than have a loop. @nchammas did some work on installing pssh on the driver node if I am not wrong

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also another minor comment is that i see warning messages of the form

No Match for argument: httpd*
No Match for argument: php*
No Match for argument: ganglia
No Match for argument: ganglia
No Match for argument: ganglia-gmond*
No Match for argument: ganglia-gmetad*
No Match for argument: httpd*
No Match for argument: php*
No Match for argument: ganglia*
No Match for argument: ganglia*
No Match for argument: ganglia-gmond*
No Match for argument: ganglia-gmetad*

Is this expected ?

ssh -t -t $SSH_OPTS root@$node "yum install -q -y $GANGLIA_PACKAGES" & sleep 0.3
done
wait

Expand Down
2 changes: 1 addition & 1 deletion templates/etc/httpd/conf/httpd.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,7 @@ LoadModule version_module modules/mod_version.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
LoadModule php5_module modules/libphp-5.5.so
LoadModule php5_module modules/libphp-5.6.so

#
# The following modules are not loaded by default:
Expand Down