DIRAC v7r0

DIRACOS

As of v7r0, Externals and Lcgbundle packages are replaced by DIRACOS. Note that runit, as well as mySQL, are not anymore distributed, and you must install them by yourself. RPMs are provided here: http://diracproject.web.cern.ch/diracproject/rpm/

Pilot 3

The "Pilot 3" becomes the default Pilot version. Technically, it means that:

• the Pilot code will now be taken from https://github.com/DIRACGrid/Pilot (the location of "Pilot2 code" is instead https://github.com/DIRACGrid/DIRAC/tree/rel-v6r22/WorkloadManagementSystem/PilotAgent)
• if you have an extension of the Pilot2 code, you will need to move it to a new repository, normally called "[YourCommunity]Pilot". For an example, please look at https://gitlab.cern.ch/lhcb-dirac/LHCbPilot
• follow doc at [add link]
• if you really want to keep using the Pilot2 code (and maybe its extension) you can keep doing it by setting the SiteDirector flag "Pilot3 = False" [add link]
  • The pilot2 code will be removed in a future DIRAC release, and it's, from now on, legacy unsupported code.

Core

it is now possible to use M2Crypto instead of PyGSI by setting the environment variable DIRAC_USE_M2CRYPTO=Yes in the bashrc. This is a technology preview as described here https://dirac.readthedocs.io/en/integration/AdministratorGuide/technologyPreviews.html

AccountingSystem

PR https://github.com/DIRACGrid/DIRAC/pull/3701 merges the ObjectLoaders and TypeLoaders used in AccountingSystem and MonitoringSystem into a single one. Pay attention in case you use them.

ALTER the AccountingDB tables that use "MinorStatus", so that it is a `VARCHAR(256). Some examples follow (adapt them to your installation):

ALTER TABLE `ac_key_DIRAC-Certification_Job_FinalMinorStatus` MODIFY value VARCHAR(256) NOT NULL;

WorkloadManagementSystem

Jobs parameters on ElasticSearch backend

• The jobs status is now kept in a separate MySQL table. Do add this table by issuing the following:

USE JobDB;

CREATE TABLE `JobsStatus` (
  `JobID` INT(11) UNSIGNED NOT NULL,
  `Status` VARCHAR(32) NOT NULL DEFAULT 'Received',
  `MinorStatus` VARCHAR(128) NOT NULL DEFAULT 'Unknown',
  `ApplicationStatus` VARCHAR(255) DEFAULT 'Unknown',
  PRIMARY KEY (`JobID`),
  FOREIGN KEY (`JobID`) REFERENCES `JobJDLs`(`JobID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

• It's possible (and advised) to store the Jobs parameters on ElasticSearch (up to now, they have been stored in a key/value pair MySQL table, part of JobDB).

TransformationSystem

PR https://github.com/DIRACGrid/DIRAC/pull/3703 replaces the TransformationInputDataQuery table with a new table TransformationMetaQueries, which stores both Input and Output MetaQueries. In order to guarantee backward compatibility, before deploying the release, it's necessary to rename the TransformationInputDataQuery table and create a view as following:

  ALTER TABLE `TransformationInputDataQuery` RENAME `TransformationMetaQueries`, 
  ADD COLUMN QueryType VARCHAR(8), 
  CHANGE COLUMN `ParameterName` `MetaDataName` VARCHAR(512) NOT NULL, 
  CHANGE COLUMN `ParameterValue` `MetaDataValue` blob NOT NULL,  
  CHANGE COLUMN `ParameterType` `MetaDataType` VARCHAR(8) NOT NULL;
  CREATE OR REPLACE VIEW `TransformationInputDataQuery` AS 
  SELECT TransformationID, 
  MetaDataName as ParameterName, 
  MetaDataValue as ParameterValue,  
  MetaDataType as ParameterType
  from TransformationMetaQueries;

ProductionSystem

With v7r0, with PR https://github.com/DIRACGrid/DIRAC/pull/3703, a completely new system is put in place: the ProductionSystem, which is used for high-level productions management.

Install the DB with:

CREATE DATABASE ProductionDB;

To install the Production System service from the admin console:

install service Production ProductionManager

DataManagementSystem

Removed old FTS system

The old FTS system has been completely removed.

Removed SRM2Storage and XROOTStorage plugins

These plugins are superseeded for a long time by GFAL2_SRM2Storage and GFAL2_XROOTStorage