We have a daemon that runs a Cascading job on request from a client. Since Cascading is build on top of Hadoop, the command to start the daemon looks like any other hadoop request.
The problem was getting the daemon to do a rotating appender in log4j. Since it is a daemon we want it to run lights out 24/7 so just redirecting stdin and stderr to a log file wasn't going to work.
So we wrote a script that sets the following environment variables before launching the daemon:
export HADOOP_LOG_DIR="/home/hadoop/mapr/hadoop_reporting/logs"
export HADOOP_LOGFILE="daemon.log"
export HADOOP_ROOT_LOGGER="INFO,console,DRFA"
The key change is the 'DRFA' line on the logger. You could ignore the first two and the logs would go to the default Hadoop location ($HADOOP_HOME/logs).
This is really useful since we can run different daemons, each writing to their own log files.
Note that this doesn't change the cluster-side logging. They all still go to where they are expected.
