Skip to content

toddlipcon/hadoop-lzo-packager

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

37 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Overview
===================
This project is a convenient mechanism of packaging the
hadoop-gpl-compression project from Google Code.

It has three basic steps:

1) Perform an svn export of the most recent revision of hadoop-gpl-compression
2) Create and build an RPM
3) Create an build a Debian package

Requirements:
====================

- subversion
- java (preferably sun's JDK)
  - JAVA_HOME must be set in your environment
- appropriate package building tools and lzo libs for your platform
  - yum install rpm-build lzo-devel         (RedHat based)
  - apt-get install dev-scripts liblzo2-dev (Debian based)
- ant version 1.7.0 or greater (RedHat will require some fiddling[1])

When you try to build for your platform, build dependency errors will also
inform you of any other packages you may need to install (eg lzo2 devel
packages, ant, etc)

[1]: First you'll need to install "ant" and "ant-nodeps" with yum.  Then
You'll need to download the binary Apache Ant distribution from
their website, and extract the tarball somewhere.  You'll have to set
ANT_HOME in your environment to point to the newly archived directory.
You'll also have to put $ANT_HOME/bin in your path, before /usr/bin.
Running "ant" on the command line should run $ANT_HOME/bin/ant, not
/usr/bin/ant.

Usage:
====================
To build packages, simply run the included shell script.

  ./run.sh

We recommend you run this on the same platform as your tasktrackers so
as to be sure the built libraries are compatible.

Various options are available, to get help do:

  ./run.sh -h

If you would like to skip building debian or rpm, you can do:

  ./run.sh --no-rpm
    or
  ./run.sh --no-deb

If you'd like to check out a particular revision, you can do:

  ./run.sh --svn-rev=46

If the downloads fail because of certificate problems, you can do:

  WGET_OPTS=--no-check-certificate ./run.sh

If the build fails and you find a file build/master then you have a
version of wget which does not use the filename from the redirected URL.
You can work around it with:

  WGET_OPTS=--trust-server-names=on ./run.sh

Or with both options:

  WGET_OPTS="--no-check-certificate --trust-server-names=on" ./run.sh

There are some other variables that can be overridden - simply look
at the top section of run.sh to learn what they are.

After running the script, you should be able to find debs in the build/deb
directory and RPMs in the build/topdir/RPMS directory.


Contributing
===================
To contribute to this project, please clone its repository from
http://github.com/toddlipcon/hadoop-lzo-packager/
and commit patches to your github repository. When you would like to
submit your contribution for inclusion, send a Pull Request to the
Cloudera repository.

About

Packaging utilities for GPL compression libraries in Hadoop

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages