ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo

How about now

?
?
D
DreamMalar

The document describes the configuration details of a scheduler that runs jobs to export social media data from various sources to a universal profile database. It outlines the deployment environment, database connections, configurable jobs, and MySQL tables used to track scheduled jobs and export status. The system job checks for new jobs every 20 minutes and exports occur based on cron expressions for each job configured in the SCHEDULABLE_JOB_TBL. Failure details are captured in MongoDB collections.

1 of 4
Download to read offline
Scheduler Architecture 1. Deployment details :- For a QA system the scheduler instance is running on DEV1 server box. ¡ö Deployment Path - /mnt1/scheduler/ ¡ö Start script location - /mnt1/scheduler/bin/ ¡ö Log File Configuration - /mnt1/scheduler/logconfig ¡ö Spring Configuration - /mnt1/scheduler/spring ¡ö Log File location - /mnt1/logs/scheduler2.0-global-logs.log ¡ö Quartz Database - All the Quartz related table are created under the following details. ¡ñ DB Host = DEV1 ¡ñ DB Name = SCHEDULER_GRL:3306 ¡ñ DB user = sprmys ¡ñ DB Password = R1k1rP5 ¡ñ Quartz Tables = QRTZ* ¡ö Job Details Database - The configured jobs and the last run details are captured in the following tables ¡ñ DB Host = QA2 ¡ñ DB Name = SPR_GRL ¡ñ SCHEDULABLE_JOB_TBL = configured jobs ¡ñ CLIENT_UNIVERSAL_PROFILE_EXPORT_TBL = details last successful runs. ¡ö Failure Details - At times the Scheduler, Job or the Trigger may fail, these details are captured in the mongo DB. ¡ñ Mongo Host = ec2-107-22-29-36.compute-1.amazonaws.com ¡ñ DB Name = SPR_GLOBAL ¡ñ Collection Names :- ¡ð failedJobDetail - Captures the failed job details ¡ð failedSchedulerDetail - Captures the failed scheduler details. ¡ð triggerMissFireDetail - Captures the missed trigger details.
2. System Job Details :- Sprinklr has been pre-configured with system job which runs every 5 minutes to check the ¡°SCHEDULABLE_JOB_TBL¡± changes in the SPR_GRL database. If this time expression to run every five minutes needs to be changed the following entry needs to be made in the ¡°spring-quartz.xml¡± file (maked in bold). Adding the following entry would ensure that the system job is scheduled to run every 20 mins as opposed to the 5 minutes it has been configured to run. Restart the scheduler instance after the change has been made to reflect the same. <bean id="sprScheduler" class="com.spr.scheduler.QuartzSchedulerImpl"> <property name="scheduler"> <ref bean="schedulerFactoryBean"/> </property> <property name="jobBasePackageName"> <value>com.spr.jobs</value> </property> <property name="overrideSystemJobMap"> <map> <entry key="com.spr.jobs.system.JobSchedulerJob"> <!-- scheduled the job to run once a century at every 20 mins --> <value>* 0/20 * * * ?</value> </entry> </map> </property> <ignored rest of the properties/>
3. Configurable Jobs :- The following list of Jobs can be configured in the scheduler. 1. com.spr.jobs.export.twitter.ExportTwitterClientProfileToUniversalProfileJob Exports the data from ¡°PROFILE_TBL¡± based on the ¡°MODIFIED_TM¡±, ¡°CLIENT_ID¡±, ¡°ACCOUNT_ID¡± and the ¡°PARTNER_ID¡± to then ¡°universalProfile¡± collection. 2. com.spr.jobs.export.twitter.ExportTwitterConversationToUniversalProfileJob Exports the data from ¡°ACCOUNT_TWEET_TBL¡± based on the ¡°CREATED_TM¡±, ¡°CLIENT_ID¡±, ¡°ACCOUNT_ID¡± and the ¡°PARTNER_ID¡± to then ¡°conversation¡± collection. 3. com.spr.jobs.export.twitter.ExportTwitterFollowerAndFollowingDetailsJob Export the data from ¡°ACCOUNT_FFUF_TBL¡± based on the ¡°MODIFIED_TM¡±, ¡°CLIENT_ID¡±, ¡°ACCOUNT_ID¡± and the ¡°PARTNER_ID¡± to the ¡°universalProfile¡± collection. 4. com.spr.jobs.export.facebook.ExportFacebookClientProfileToUniversalProfileJob Export the data from ¡°FB_PROFILE_TBL¡± based on the ¡°MODIFIED_TM¡±, ¡°CLIENT_ID¡±, ¡°ACCOUNT_ID¡± and the ¡°PARTNER_ID¡± to the universalProfile¡± collection. 5. com.spr.jobs.export.facebook.ExportFacebookCommentsToUniversalProfileJob Exports the data from ¡°FB_ACCOUNT_COMMENTS_TBL¡± based on the ¡°CREATED_TM¡±, ¡°CLIENT_ID¡±, ¡°ACCOUNT_ID¡± and the ¡°PARTNER_ID¡± to the ¡°conversation¡± collection. 6. com.spr.jobs.export.facebook.ExportFacebookConversationToUniversalProfileJob Exports the data from ¡°FB_INBOX_STREAM_TBL¡± based on the ¡°CREATED_TM¡±, ¡°CLIENT_ID¡±, ¡°ACCOUNT_ID¡± and the ¡°PARTNER_ID¡± to the ¡°conversation¡± collection.
4. MySql Tables :- ¡ñ SCHEDULABLE_JOB_TBL - This table captures the job which need to be scheduled in the scheduler. The JOB_CLASS, PARTNER_ID, CLIENT_ID and the CLIENT_ID form the uniqueness. ¡ð SCHEDULABLE_JOB_TBL_ID - The primary key ¡ð JOB_CLASS_NAME - The name of job class, as defined above. This is the business logic to executes. ¡ð PARTNER_ID - The Partner Id ¡ð CLIENT_ID - The Client Id ¡ð ACCOUNT_ID - The Account Id ¡ð JOB_DATA - Any additional data as a JSON string to be made available to job during its execution. ¡ð CRON_EXPRESSION - The Cron Expression which defines the run intervals. ¡ð IS_SCHEDULED - Boolean to define is the job is scheduled on the scheduler. This column is used by the System Job and is internal to the scheduler. ¡ð IS_ENABLED - if set to true, would enable the job in the scheduler to run on the scheduler cron time, if set to false, would disable the enabled job from the scheduler, and this job will not be qualified for any further runs. ¡ð CREATED_TM - The time at which this record was created ¡ð MODIFIED_TM - The time at which this record was updated. ¡ð DEL_FLG - If set to true, would delete the job from the scheduler along with its associated triggers. To qualify this job to run again set the flag to false. ¡ð SCHEDULE_KEY - The unique key which identifies the job. This column is used by the System Job and is internal to the scheduler. ¡ñ CLIENT_UNIVERSAL_PROFILE_EXPORT_TBL - This table captures the last run details for the job configured. This table is internal to the jobs and should be used only for look- ups. ¡ð PARTNER_ID - The Partner id ¡ð CLIENT_ID - The Client Id ¡ð ACCOUNT_ID - The account Id ¡ð LAST_EXPORT_DATE - The date till which the job did its last export. ¡ð SN_TYPE - The Social Network Type ¡ð EXPORT_TYPE - The Export type, can be one of Profile, Conversation or Comment

More Related Content

How about now

  • 1. Scheduler Architecture 1. Deployment details :- For a QA system the scheduler instance is running on DEV1 server box. ¡ö Deployment Path - /mnt1/scheduler/ ¡ö Start script location - /mnt1/scheduler/bin/ ¡ö Log File Configuration - /mnt1/scheduler/logconfig ¡ö Spring Configuration - /mnt1/scheduler/spring ¡ö Log File location - /mnt1/logs/scheduler2.0-global-logs.log ¡ö Quartz Database - All the Quartz related table are created under the following details. ¡ñ DB Host = DEV1 ¡ñ DB Name = SCHEDULER_GRL:3306 ¡ñ DB user = sprmys ¡ñ DB Password = R1k1rP5 ¡ñ Quartz Tables = QRTZ* ¡ö Job Details Database - The configured jobs and the last run details are captured in the following tables ¡ñ DB Host = QA2 ¡ñ DB Name = SPR_GRL ¡ñ SCHEDULABLE_JOB_TBL = configured jobs ¡ñ CLIENT_UNIVERSAL_PROFILE_EXPORT_TBL = details last successful runs. ¡ö Failure Details - At times the Scheduler, Job or the Trigger may fail, these details are captured in the mongo DB. ¡ñ Mongo Host = ec2-107-22-29-36.compute-1.amazonaws.com ¡ñ DB Name = SPR_GLOBAL ¡ñ Collection Names :- ¡ð failedJobDetail - Captures the failed job details ¡ð failedSchedulerDetail - Captures the failed scheduler details. ¡ð triggerMissFireDetail - Captures the missed trigger details.
  • 2. 2. System Job Details :- Sprinklr has been pre-configured with system job which runs every 5 minutes to check the ¡°SCHEDULABLE_JOB_TBL¡± changes in the SPR_GRL database. If this time expression to run every five minutes needs to be changed the following entry needs to be made in the ¡°spring-quartz.xml¡± file (maked in bold). Adding the following entry would ensure that the system job is scheduled to run every 20 mins as opposed to the 5 minutes it has been configured to run. Restart the scheduler instance after the change has been made to reflect the same. <bean id="sprScheduler" class="com.spr.scheduler.QuartzSchedulerImpl"> <property name="scheduler"> <ref bean="schedulerFactoryBean"/> </property> <property name="jobBasePackageName"> <value>com.spr.jobs</value> </property> <property name="overrideSystemJobMap"> <map> <entry key="com.spr.jobs.system.JobSchedulerJob"> <!-- scheduled the job to run once a century at every 20 mins --> <value>* 0/20 * * * ?</value> </entry> </map> </property> <ignored rest of the properties/>
  • 3. 3. Configurable Jobs :- The following list of Jobs can be configured in the scheduler. 1. com.spr.jobs.export.twitter.ExportTwitterClientProfileToUniversalProfileJob Exports the data from ¡°PROFILE_TBL¡± based on the ¡°MODIFIED_TM¡±, ¡°CLIENT_ID¡±, ¡°ACCOUNT_ID¡± and the ¡°PARTNER_ID¡± to then ¡°universalProfile¡± collection. 2. com.spr.jobs.export.twitter.ExportTwitterConversationToUniversalProfileJob Exports the data from ¡°ACCOUNT_TWEET_TBL¡± based on the ¡°CREATED_TM¡±, ¡°CLIENT_ID¡±, ¡°ACCOUNT_ID¡± and the ¡°PARTNER_ID¡± to then ¡°conversation¡± collection. 3. com.spr.jobs.export.twitter.ExportTwitterFollowerAndFollowingDetailsJob Export the data from ¡°ACCOUNT_FFUF_TBL¡± based on the ¡°MODIFIED_TM¡±, ¡°CLIENT_ID¡±, ¡°ACCOUNT_ID¡± and the ¡°PARTNER_ID¡± to the ¡°universalProfile¡± collection. 4. com.spr.jobs.export.facebook.ExportFacebookClientProfileToUniversalProfileJob Export the data from ¡°FB_PROFILE_TBL¡± based on the ¡°MODIFIED_TM¡±, ¡°CLIENT_ID¡±, ¡°ACCOUNT_ID¡± and the ¡°PARTNER_ID¡± to the universalProfile¡± collection. 5. com.spr.jobs.export.facebook.ExportFacebookCommentsToUniversalProfileJob Exports the data from ¡°FB_ACCOUNT_COMMENTS_TBL¡± based on the ¡°CREATED_TM¡±, ¡°CLIENT_ID¡±, ¡°ACCOUNT_ID¡± and the ¡°PARTNER_ID¡± to the ¡°conversation¡± collection. 6. com.spr.jobs.export.facebook.ExportFacebookConversationToUniversalProfileJob Exports the data from ¡°FB_INBOX_STREAM_TBL¡± based on the ¡°CREATED_TM¡±, ¡°CLIENT_ID¡±, ¡°ACCOUNT_ID¡± and the ¡°PARTNER_ID¡± to the ¡°conversation¡± collection.
  • 4. 4. MySql Tables :- ¡ñ SCHEDULABLE_JOB_TBL - This table captures the job which need to be scheduled in the scheduler. The JOB_CLASS, PARTNER_ID, CLIENT_ID and the CLIENT_ID form the uniqueness. ¡ð SCHEDULABLE_JOB_TBL_ID - The primary key ¡ð JOB_CLASS_NAME - The name of job class, as defined above. This is the business logic to executes. ¡ð PARTNER_ID - The Partner Id ¡ð CLIENT_ID - The Client Id ¡ð ACCOUNT_ID - The Account Id ¡ð JOB_DATA - Any additional data as a JSON string to be made available to job during its execution. ¡ð CRON_EXPRESSION - The Cron Expression which defines the run intervals. ¡ð IS_SCHEDULED - Boolean to define is the job is scheduled on the scheduler. This column is used by the System Job and is internal to the scheduler. ¡ð IS_ENABLED - if set to true, would enable the job in the scheduler to run on the scheduler cron time, if set to false, would disable the enabled job from the scheduler, and this job will not be qualified for any further runs. ¡ð CREATED_TM - The time at which this record was created ¡ð MODIFIED_TM - The time at which this record was updated. ¡ð DEL_FLG - If set to true, would delete the job from the scheduler along with its associated triggers. To qualify this job to run again set the flag to false. ¡ð SCHEDULE_KEY - The unique key which identifies the job. This column is used by the System Job and is internal to the scheduler. ¡ñ CLIENT_UNIVERSAL_PROFILE_EXPORT_TBL - This table captures the last run details for the job configured. This table is internal to the jobs and should be used only for look- ups. ¡ð PARTNER_ID - The Partner id ¡ð CLIENT_ID - The Client Id ¡ð ACCOUNT_ID - The account Id ¡ð LAST_EXPORT_DATE - The date till which the job did its last export. ¡ð SN_TYPE - The Social Network Type ¡ð EXPORT_TYPE - The Export type, can be one of Profile, Conversation or Comment