- Login or register to hide this ad -

Scheduling

From DHIS2Academy

Jump to: navigation, search

Contents

Introduction

Group members

  • Thomas Torp (thomtor@ifi.uio.no)
  • Joshi Suhas Govind (joshi@ifi.uio.no)
  • Jonathan Lunde Lillesæter (jonathal@ifi.uio.no)
  • Christian Tryti (cotryti@ifi.uio.no)

Background

In India the server solution is running tasks on one server per province. Some of the tasks running are very CPU-intensive. These tasks affect the rest of the system's performance and as a result other user will experience a lower responsiveness. This should preferably be avoided. The missing distribution of system load creates a necessity for task scheduling. Heavy tasks should be scheduled to run overnight when the server capacity is otherwise not occupied. This way users accessing the system during daytime would not be affected by CPU-intensive tasks.

Project presentation

We presented our project on the group presentation November 25th.


Requirements and roadmap

Detailed requirements

To set up a proper scheduler, we need to implement the following requirements:


By dividing our main goal into several sub-goals, we will be able to test and evaluate these features individually. The purpose of the optional goal is to calculate the time of each task scheduled. This way we can get a weighted average run time and perhaps use that later for optimizing queuing functionality. Also, if tasks could be identified at some point, previous run time on given task could be used to present a statistical estimate of expected run time, which could come in very handy.


Roadmap and milestones

Milestone 1 - October 22nd:


Milestone 2 - November 5th:


Milestone 3 - November 26th:


Milestone 4 - December 10th:


Technical challenges

These topics were addressed (and somewhat discussed) during the project presentation. Our respective solutions for these challenges are presented in the next section Evaluation of methods.

Which scheduling interface is best suited?

We had a few option when selecting which interface to implement.

  • What was considered?

Look here for information about this

  • Why was ThreadPoolTaskSchedulerselected?

Look here for information about this

  • ScheduleFuture-problem

The main problem about the ScheduledFuture object is the lack of information we can get from it. Its just seven methods to use and the main methods is isDone(), isCancelled() and cancel(). So to gather any information about the object we need a wrapper. This is the only way to be able to edit task if you want to add for example a new date. So until oracle decides to create a better API, are we think that this is the way to do it.

How do we intercept methods?

This was the most challenging part of the project. Read more: Logging and Interception

Assumption

We are making the following assumption: "Task runtime will never be more than 1 hour"

Since we have no way of knowing how long a task will run, we cannot know exactly when our queue is open or not. We are restricted by the following limitation:

  • We have no background information about the task, nor prior knowledge about runtime
  • We cannot get any feedback on task progress, only check to see if finished or not
  • We cannot pause tasks or store them, so they cannot be paused or reschedueled once started

We have been told that these tasks shouldn't take more than approximatly one hour, so we haven't implemented any technical way of handling overlapping scheduling. Some of these issues are however addressed in our optional goal of timing tasks.


Evaluation of methods

Scheduling

We started with looking on how to define tasks in in Spring. The first thing we found was task executor. This is able to execute task that implements Runnable.

The problem with this approach is that we need to implement the scheduling part. So this we can work with, but will be a lot of work.

The first scheduling interface we found was ScheduledTimerTask. This was close to what we wanted but the real problem with ScheduledTimerTask is that it cant take a given date or if it should repeat the task. In the API it said "The J2SE Timer does not offer more sophisticated scheduling options like cron expressions. Consider using Quartz for such demanding needs." So we started searching around for cron expressions.

After a while we found some great documentation on scheduling, here.

public interface TaskScheduler {
    ScheduledFuture schedule(Runnable task, Trigger trigger);
    ScheduledFuture schedule(Runnable task, Date startTime);
    ScheduledFuture scheduleAtFixedRate(Runnable task, Date startTime, long period);
    ScheduledFuture scheduleAtFixedRate(Runnable task, long period);
    ScheduledFuture scheduleWithFixedDelay(Runnable task, Date startTime, long delay);
    ScheduledFuture scheduleWithFixedDelay(Runnable task, long delay);
}

This is what we want, and what we are going to use in our project. It is three implementations of the interface: ConcurrentTaskScheduler, ThreadPoolTaskScheduler and TimerManagerTaskScheduler.

We ended up with ThreadPoolTaskScheduler because it is the default implementation of the TaskScheduler interface.


Logging and interception

Our scheduling implementation keeps tasks in 3 separate lists: a list for currently scheduled tasks, currently running tasks and completed tasks. In order to maintain these lists, and to log interesting statistical data, the scheduler needs to be notified that a task has started or completed running.

Alternative approaches

Create a ScheduledTask interface

Create an interface that specifies what the user (those who want to schedule a task) needs to implement. This would need to vary between schedule implementations.

  • + No extra work on our part
  • - Shifts the responsibility over to the user (to reschedule, , keep track of time)


Wrap a Runnable around the task

The only requirement needed for scheduling a task is that the task implements the Runnable interface. We could insert the users task into our own Runnable object, that when started, calls the appropriate 'before' methods, starts the task, and when the task returns, call the appropriate 'after' methods.

  • + Quick and easy to implement
  • More of a work-around than a solution


External MethodInterceptor

Create the interceptor as a stand-alone object, seperate of the scheduling implementation.

  • + Can easly be switched out for another interceptor implementation
  • - No access to the context around the intercepted method; we only know the method was run, not which task was run. This is a problem since we need to edit variables in the scheduler whenever a task starts or stops running.


Internal MethodInterceptor

Create the interceptor as an internal integrated anonymous class.

  • + It is declared as an inner class so we have access to methods that we need that aren't declared in the Scheduler-interface.
  • Integrated into the implementation.


What is implemented

We ended up implementing the internal interceptor. Because the maintenance of the lists and the time logging are so closely tied to the implementation, it's very convenient having the interceptor as closely tied. We don't see any situation where anyone would want to use this interceptor for anything else, or use a different interceptor-implementation.

Here is the method for creating the proxy:

    private Runnable createProxyTask( Runnable task, final int id )
    {
        ProxyFactory pf = new ProxyFactory( task );
        pf.addAdvice( new MethodInterceptor()
        {
            int _id = id;

            @Override
            public Object invoke( MethodInvocation mi )
                throws Throwable
            {
                if ( mi.getMethod().getName().equals( "run" ) )
                {
                    before();
                    mi.proceed();
                    after();
                } else 
                    mi.proceed();
                return null;
            }

            private void before()
            {
                moveTaskToRunningList( _id );
                logStartTime( _id );
            }

            private void after()
            {
                logEndTime( _id );
                moveTaskToDoneOrScheduledList( _id );
            }
        } );

        return (Runnable) pf.getProxy();
    }


Documentation

Developer documentation

User documentation


Known issues and future work

  • Fix overlapping scheduling due to time conflicts
  • More sanitization of input data
  • Fix date-in-past-problem (doesn't get rescheduled properly)
  • Resizable text box for job descriptions (probably exists in JQuery)
  • Localization support for date/time


Resources

Canceling a Scheduled Future

Crontriggers to ThreadPoolTaskScheduler

Velocity user guide

ScheduledFuture

Spring Scheduling reference

ConcurrentTaskScheduler

ThreadPoolTaskScheduler

TimerManagerTaskScheduler


Feedback

<You will receive feedback from course administration here>

M1

Approved.

- Morten

M2

Approved. Good work.

- Morten

M3

Approved.

- Morten

M4

Approved.

- Morten

Personal tools