Parallel jobs using multiple queues

By default NetYCE uses a single queue, which can run up to 50 jobs in parallel, with a 2 second schedule interval.

Example: when you schedule 50 jobs with 1 queue it takes up to a total of 100 seconds for all the jobs to be scheduled.

If you desire to schedule multiple jobs at the same time, you could increase the amount of queues.

Example: when you schedule 50 jobs with 2 queues it takes up to a total of 50 seconds for all the jobs to be scheduled.

When the jobs are scheduled the job execution will always be parallel. This configuration change will only speed-up the process of scheduling multiple jobs at the same time.

The configuration below is an example with 5 queues. With this setup you can schedule 50 jobs in 20 seconds, assuming they are evenly spread (10 jobs per queue).

The tool_setup.xml file is configured with additional queues:

    <queues>
        <queue name="yce"  done_age="180" cancel_age="1800" job_int="2" max_run="50" max_wait="3600" />
        <queue name="yce2" done_age="180" cancel_age="1800" job_int="2" max_run="10" max_wait="3600" />
        <queue name="yce3" done_age="180" cancel_age="1800" job_int="2" max_run="10" max_wait="3600" />
        <queue name="yce4" done_age="180" cancel_age="1800" job_int="2" max_run="10" max_wait="3600" />
        <queue name="yce5" done_age="180" cancel_age="1800" job_int="2" max_run="10" max_wait="3600" />
    </queues>

The API call can specify the queue:

<task>
	<head passwd="password" req_app="api_call" task_name="command_job" task_type="xml_request" userid="user" />
	<request 
		node_name="nodeA"
		stored_job_name="ntp_server"
		description="Fix NTP server"
		evaluate="no"
		sched_day="tomorrow"
		sched_now="no"
		sched_time="5:05"
		verbose_log="yes"
		sched_queue="yce"
		/>
</task>

Here, the sched_queue can be manipulated to choose any of the other specified queues. This will work best if the jobs are scheduled in the future (sched_now=“no”), therefore have the parsing done for all jobs being added to the queues. At the specified time all queues are read and the jobs are scheduled.

Be aware, that depending on the sizing of the server, the usage of complex relations and variable constructions and the amount of jobs you wish to push this could impact the responsiveness on the server.

More detail for each of the parts can be found at their respective pages: