fr.gouv.culture.sdx.oai
Class AbstractDocumentBaseOAIHarvester

java.lang.Object
  extended byfr.gouv.culture.util.apache.avalon.excalibur.concurrent.Semaphore
      extended byfr.gouv.culture.util.apache.avalon.excalibur.concurrent.Mutex
          extended byfr.gouv.culture.util.apache.cocoon.xml.AbstractSynchronizedXMLProducer
              extended byfr.gouv.culture.util.apache.cocoon.xml.AbstractSynchronizedXMLPipe
                  extended byfr.gouv.culture.oai.SynchronizedOAIObjectImpl
                      extended byfr.gouv.culture.oai.AbstractOAIHarvester
                          extended byfr.gouv.culture.sdx.oai.AbstractDocumentBaseOAIHarvester
All Implemented Interfaces:
org.apache.avalon.framework.component.Composable, org.apache.avalon.framework.configuration.Configurable, org.xml.sax.ContentHandler, org.xml.sax.ext.LexicalHandler, org.apache.avalon.framework.logger.LogEnabled, OAIHarvester, OAIObject, org.apache.avalon.excalibur.pool.Poolable, org.apache.avalon.excalibur.pool.Recyclable, org.apache.avalon.excalibur.concurrent.Sync, SynchronizedXMLConsumer, SynchronizedXMLPipe, SynchronizedXMLProducer, Target, org.apache.cocoon.xml.XMLConsumer, org.apache.avalon.excalibur.xml.XMLizable, org.apache.cocoon.xml.XMLPipe, org.apache.cocoon.xml.XMLProducer
Direct Known Subclasses:
LuceneDocumentBaseOAIHarvester

public abstract class AbstractDocumentBaseOAIHarvester
extends AbstractOAIHarvester
implements Target

Created by IntelliJ IDEA. User: rpandey Date: May 12, 2003 Time: 12:32:16 PM


Nested Class Summary
 
Nested classes inherited from class fr.gouv.culture.oai.OAIObject
OAIObject.Node
 
Field Summary
protected static java.lang.String ATTRIBUTE_NAME_ADMIN_EMAIL
          Configuration node name
protected static java.lang.String ATTRIBUTE_NAME_FROM
          Configuration node name
protected static java.lang.String ATTRIBUTE_NAME_KEEP_DELETED_RECORD
           
protected static java.lang.String ATTRIBUTE_NAME_METADATA_PREFIX
          Configuration node name
protected static java.lang.String ATTRIBUTE_NAME_NAME
          Configuration node name
protected static java.lang.String ATTRIBUTE_NAME_SDX_REPOSITORY
          Configuration node name
protected static java.lang.String ATTRIBUTE_NAME_SET
          Configuration node name
protected static java.lang.String ATTRIBUTE_NAME_UNTIL
          Configuration node name
protected static java.lang.String ATTRIBUTE_NAME_UPDATE
          Configuration node name
protected static java.lang.String ATTRIBUTE_NAME_URL
          Configuration node name
protected static java.lang.String ATTRIBUTE_NAME_USER_AGENT
          Configuration node name
protected static java.lang.String ATTRIBUTE_NO_RECORDS_PER_BATCH
          Configuration node name
protected static java.lang.String ATTRIBUTE_USE_LAST_HARVEST_DATE
          Configuration node name
protected  Database database
          Underlying database to store any info
protected  java.util.ArrayList deletedDocs
           
protected  DocumentBase docbase
          The underlying document base
protected  java.lang.String docbaseId
          Id of the underlying document base
protected  java.util.Hashtable docBaseProps
          Properties from the document base
protected static java.lang.String ELEMENT_NAME_OAI_DATA_PROVIDERS
          Configuration node name
protected static java.lang.String ELEMENT_NAME_OAI_IDENTIFIER
          Configuration node name
protected static java.lang.String ELEMENT_NAME_OAI_VERB
          Configuration node name
protected  java.lang.String ELEMENT_NAME_PIPELINE
           
protected  java.io.FileOutputStream fileOs
           
protected  java.util.Hashtable filesProperties
          List OAI files with OAI properties
protected  java.io.File harvestDoc
           
protected  java.util.ArrayList harvestedDocs
           
protected  IDGenerator harvesterIdGen
          IDGenerator for this object
protected  boolean keepDeletedRecords
           
protected static java.lang.String NO_DOCS_DELETED
           
protected static java.lang.String NO_DOCS_HARVESTED
           
protected  int noDocsDeleted
           
protected  int noDocsHarvested
           
protected  int noRecordsPerBatch
           
protected static java.lang.String OAI_FAILED_HARVEST
           
protected static java.lang.String OAI_FROM
           
protected static java.lang.String OAI_HARVEST_ID
           
protected static java.lang.String OAI_HARVESTER_LAST_UPDATED
           
protected static java.lang.String OAI_HARVESTER_RESUMPTION_TOKEN
           
protected static java.lang.String OAI_IDENTIFIER
           
protected static java.lang.String OAI_METADATA_PREFIX
           
protected static java.lang.String OAI_SET
           
protected static java.lang.String OAI_UNTIL
           
protected static java.lang.String OAI_VERB
           
protected  Pipeline pipe
          Pre-indexation pipeline
protected  TimeScheduler scheduler
          Time scheduler for stored requests
protected  java.util.Hashtable storedRequests
          Requests in application.xconf
protected  java.util.Hashtable storeRepositoriesRefs
          References to the underlying documentbase's/application's repositories
protected  java.io.File tempDir
           
protected  java.lang.String TEMPFILE_SUFFIX
           
protected  XMLDocument urlResource
           
 
Fields inherited from class fr.gouv.culture.oai.AbstractOAIHarvester
adminEmails, captureElemContent, captureRecord, currentDatestamp, currentMetadtaUrlIdentifier, currentOaiIdentifier, currentOaiStatus, deleteRecord, firstXmlConsumer, identifierName, manager, newRequestUrl, OAI_REPOSITORY_URL, OAI_REQUEST_URL, repoUrl, requestParams, requestUrl, responseDate, resumptionToken, sBuff, userAgent
 
Fields inherited from class fr.gouv.culture.oai.SynchronizedOAIObjectImpl
logger
 
Fields inherited from class fr.gouv.culture.util.apache.cocoon.xml.AbstractSynchronizedXMLPipe
synchronizedXmlConsumerAcquired
 
Fields inherited from class fr.gouv.culture.util.apache.cocoon.xml.AbstractSynchronizedXMLProducer
synchronizedXmlConsumer
 
Fields inherited from interface fr.gouv.culture.oai.OAIObject
HTTP_HEADER_NAME_FROM, HTTP_HEADER_NAME_USER_AGENT, NUMBER_RECORDS_PER_RESPONSE
 
Constructor Summary
AbstractDocumentBaseOAIHarvester(DocumentBase base)
          Basic constructor
 
Method Summary
protected  void captureRecord()
          Ends the capture of an oai record.
protected  void captureResourceFromUrlIdentifier()
          Captures the xml from a url taken from an oai record and adds it to the oai-record as a sibling of the element
 void configure(org.apache.avalon.framework.configuration.Configuration configuration)
          Configures this object
protected  void configureAdminEmails(org.apache.avalon.framework.configuration.Configuration configuration)
          Configures a list of admin emails can be sub-elements, a single attribute, or both
protected  void configureDatabase(org.apache.avalon.framework.configuration.Configuration configuration)
          Configures the internal database
protected  void configureDataProviders(org.apache.avalon.framework.configuration.Configuration configuration)
          Configures data providers info that can be reused and from which requests can be automatically executed
protected  void configureHarvestIDGenerator(org.apache.avalon.framework.configuration.Configuration configuration)
          Configures the id generator for harvests
protected  void configurePipeline(org.apache.avalon.framework.configuration.Configuration configuration)
          Configures the preIndexation pipeline
protected  void configureStoreRepositories(java.lang.String repoUrl, org.apache.avalon.framework.configuration.Configuration oaiRepoConf)
          Configures the repositories to which data will be stored based upon their repository url
protected  void configureUpdateTriggers(java.lang.String requestUrl, org.apache.avalon.framework.configuration.Configuration updateConf)
          Configures time triggers for stored requests
protected  void deleteOAIDocuments()
          Delete OAI documents from the current document base.
protected  void deleteTempDir()
          Deletes the directory represented by the tempDir class field
 void endElement(java.lang.String s, java.lang.String s1, java.lang.String s2)
          Receive notification of the end of an element.
protected  java.lang.String generateNewHarvestId()
          Generates an id to associate with a harvest
protected  java.lang.String getHarvesterId()
          Returns an id for this harvester based upon the underlying document base id
protected  IndexParameters getIndexParameters()
          Builds simple index parameters for indexation of oai records into the undelryi
protected  java.lang.String getIsoDate()
          Get's the current date in iso8601 format
protected  java.io.File getNewTempDir()
          Creates a new temporary directory for writing harvested records before the will be indexed
protected  void handleResumptionToken()
          Handles the resumption token by issuing another request based upon the request from which the resumption token was received.
protected  void initTempDir()
          Establishes the tempDir class field
 java.util.Date lastUpdated()
          Retrieves the time when the harvester was last updated
protected  void prepareRecordCapture()
          Sets up resources to capture an oai record
protected  void prepareRecordForDeletion()
          Sets up resources to delete an oai record Add the record to the list of the records to removed
protected  void prepareResourceFromUrlIdentifierCapture()
          Prepares to read a url value from an oai record and retrieve the XML behind.
 void purgePastHarvestsData()
          Destroys all summary data pertaining to past harvests but not the actual oai records harvested
protected  void resetAllFields()
          Resets necessary class fields
protected  void resetRecordCaptureFields(boolean deleteDoc)
          Resets the class fields for record capture possibility deleting the current harvetDoc object underlying file
protected  void saveCriticalFields(boolean dataHarvested)
          Saves critical data about a harvest
 void sendPastHarvestsSummary()
          Sends sax events to the current consumer with summary details of the all the past harvests
 void sendStoredHarvestingRequests()
          Sends the details of stored harvesting requests to the current consumer
 void setProperties(java.util.Hashtable props)
          Set's the properties for this object
protected  boolean shouldHarvestDocument()
          Querys the underlying data structures based upon current sax flow position/set class fields and determines whether an oai record should be harvested
 void startElement(java.lang.String s, java.lang.String s1, java.lang.String s2, org.xml.sax.Attributes attributes)
          Receive notification of the beginning of an element.
protected  void storeFailedHarvestData(java.lang.Exception e)
          Stores data about harvesting failures caused by problems other than oai errors sent from a queried repository
protected  boolean storeHarvestedData()
          Reads the documents from tempDir and indexes them in the corresponding document base, any marked deletions will be carried out as well
 void targetTriggered(java.lang.String triggerName)
          Triggers a oai request to a repository based upon a trigger name (also a request url)
 
Methods inherited from class fr.gouv.culture.oai.AbstractOAIHarvester
abortRecordCapture, characters, compose, getAdminEmails, getHarvestParameters, handleErrors, receiveRequest, receiveSynchronizedRequest, receiveSynchronizedRequest, recycle, resetAllFieldsFinally, resetResumptionToken, setAdminEmails, setConsumer, setIdentifierName, toSAX
 
Methods inherited from class fr.gouv.culture.oai.SynchronizedOAIObjectImpl
enableLogging, sendElement, sendElementContent
 
Methods inherited from class fr.gouv.culture.util.apache.cocoon.xml.AbstractSynchronizedXMLPipe
acquireSynchronizedXMLConsumer, comment, endCDATA, endDocument, endDTD, endEntity, endPrefixMapping, ignorableWhitespace, processingInstruction, releaseSynchronizedXMLConsumer, setDocumentLocator, skippedEntity, startCDATA, startDocument, startDTD, startEntity, startPrefixMapping
 
Methods inherited from class fr.gouv.culture.util.apache.cocoon.xml.AbstractSynchronizedXMLProducer
setConsumer, setSynchronizedConsumer
 
Methods inherited from class fr.gouv.culture.util.apache.avalon.excalibur.concurrent.Mutex
isAcquired
 
Methods inherited from class fr.gouv.culture.util.apache.avalon.excalibur.concurrent.Semaphore
acquire, attempt, getTokens, release
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface org.apache.avalon.framework.logger.LogEnabled
enableLogging
 
Methods inherited from interface org.xml.sax.ContentHandler
endDocument, endPrefixMapping, ignorableWhitespace, processingInstruction, setDocumentLocator, skippedEntity, startDocument, startPrefixMapping
 
Methods inherited from interface org.xml.sax.ext.LexicalHandler
comment, endCDATA, endDTD, endEntity, startCDATA, startDTD, startEntity
 
Methods inherited from interface fr.gouv.culture.util.apache.cocoon.xml.SynchronizedXMLProducer
setConsumer
 
Methods inherited from interface org.apache.avalon.excalibur.concurrent.Sync
acquire, attempt, release
 

Field Detail

ELEMENT_NAME_PIPELINE

protected java.lang.String ELEMENT_NAME_PIPELINE

docbase

protected DocumentBase docbase
The underlying document base


docbaseId

protected java.lang.String docbaseId
Id of the underlying document base


docBaseProps

protected java.util.Hashtable docBaseProps
Properties from the document base


pipe

protected Pipeline pipe
Pre-indexation pipeline


database

protected Database database
Underlying database to store any info


storedRequests

protected java.util.Hashtable storedRequests
Requests in application.xconf


storeRepositoriesRefs

protected java.util.Hashtable storeRepositoriesRefs
References to the underlying documentbase's/application's repositories


scheduler

protected TimeScheduler scheduler
Time scheduler for stored requests


harvesterIdGen

protected IDGenerator harvesterIdGen
IDGenerator for this object


TEMPFILE_SUFFIX

protected java.lang.String TEMPFILE_SUFFIX

tempDir

protected java.io.File tempDir

harvestDoc

protected java.io.File harvestDoc

fileOs

protected java.io.FileOutputStream fileOs

urlResource

protected XMLDocument urlResource

deletedDocs

protected java.util.ArrayList deletedDocs

harvestedDocs

protected java.util.ArrayList harvestedDocs

noDocsHarvested

protected int noDocsHarvested

noDocsDeleted

protected int noDocsDeleted

keepDeletedRecords

protected boolean keepDeletedRecords

noRecordsPerBatch

protected int noRecordsPerBatch

ELEMENT_NAME_OAI_DATA_PROVIDERS

protected static final java.lang.String ELEMENT_NAME_OAI_DATA_PROVIDERS
Configuration node name

See Also:
Constant Field Values

ELEMENT_NAME_OAI_VERB

protected static final java.lang.String ELEMENT_NAME_OAI_VERB
Configuration node name

See Also:
Constant Field Values

ELEMENT_NAME_OAI_IDENTIFIER

protected static final java.lang.String ELEMENT_NAME_OAI_IDENTIFIER
Configuration node name

See Also:
Constant Field Values

ATTRIBUTE_NAME_NAME

protected static final java.lang.String ATTRIBUTE_NAME_NAME
Configuration node name

See Also:
Constant Field Values

ATTRIBUTE_NAME_ADMIN_EMAIL

protected static final java.lang.String ATTRIBUTE_NAME_ADMIN_EMAIL
Configuration node name

See Also:
Constant Field Values

ATTRIBUTE_NAME_USER_AGENT

protected static final java.lang.String ATTRIBUTE_NAME_USER_AGENT
Configuration node name

See Also:
Constant Field Values

ATTRIBUTE_NAME_URL

protected static final java.lang.String ATTRIBUTE_NAME_URL
Configuration node name

See Also:
Constant Field Values

ATTRIBUTE_NAME_UPDATE

protected static final java.lang.String ATTRIBUTE_NAME_UPDATE
Configuration node name

See Also:
Constant Field Values

ATTRIBUTE_NAME_METADATA_PREFIX

protected static final java.lang.String ATTRIBUTE_NAME_METADATA_PREFIX
Configuration node name

See Also:
Constant Field Values

ATTRIBUTE_NAME_SDX_REPOSITORY

protected static final java.lang.String ATTRIBUTE_NAME_SDX_REPOSITORY
Configuration node name

See Also:
Constant Field Values

ATTRIBUTE_NAME_FROM

protected static final java.lang.String ATTRIBUTE_NAME_FROM
Configuration node name

See Also:
Constant Field Values

ATTRIBUTE_USE_LAST_HARVEST_DATE

protected static final java.lang.String ATTRIBUTE_USE_LAST_HARVEST_DATE
Configuration node name

See Also:
Constant Field Values

ATTRIBUTE_NAME_UNTIL

protected static final java.lang.String ATTRIBUTE_NAME_UNTIL
Configuration node name

See Also:
Constant Field Values

ATTRIBUTE_NAME_SET

protected static final java.lang.String ATTRIBUTE_NAME_SET
Configuration node name

See Also:
Constant Field Values

ATTRIBUTE_NAME_KEEP_DELETED_RECORD

protected static final java.lang.String ATTRIBUTE_NAME_KEEP_DELETED_RECORD
See Also:
Constant Field Values

ATTRIBUTE_NO_RECORDS_PER_BATCH

protected static final java.lang.String ATTRIBUTE_NO_RECORDS_PER_BATCH
Configuration node name

See Also:
Constant Field Values

OAI_HARVEST_ID

protected static final java.lang.String OAI_HARVEST_ID
See Also:
Constant Field Values

OAI_FAILED_HARVEST

protected static final java.lang.String OAI_FAILED_HARVEST
See Also:
Constant Field Values

OAI_HARVESTER_LAST_UPDATED

protected static final java.lang.String OAI_HARVESTER_LAST_UPDATED
See Also:
Constant Field Values

OAI_HARVESTER_RESUMPTION_TOKEN

protected static final java.lang.String OAI_HARVESTER_RESUMPTION_TOKEN
See Also:
Constant Field Values

OAI_VERB

protected static final java.lang.String OAI_VERB
See Also:
Constant Field Values

OAI_IDENTIFIER

protected static final java.lang.String OAI_IDENTIFIER
See Also:
Constant Field Values

OAI_METADATA_PREFIX

protected static final java.lang.String OAI_METADATA_PREFIX
See Also:
Constant Field Values

OAI_FROM

protected static final java.lang.String OAI_FROM
See Also:
Constant Field Values

OAI_UNTIL

protected static final java.lang.String OAI_UNTIL
See Also:
Constant Field Values

OAI_SET

protected static final java.lang.String OAI_SET
See Also:
Constant Field Values

NO_DOCS_DELETED

protected static final java.lang.String NO_DOCS_DELETED
See Also:
Constant Field Values

NO_DOCS_HARVESTED

protected static final java.lang.String NO_DOCS_HARVESTED
See Also:
Constant Field Values

filesProperties

protected java.util.Hashtable filesProperties
List OAI files with OAI properties

Constructor Detail

AbstractDocumentBaseOAIHarvester

public AbstractDocumentBaseOAIHarvester(DocumentBase base)
Basic constructor

Method Detail

setProperties

public void setProperties(java.util.Hashtable props)
Set's the properties for this object


configure

public void configure(org.apache.avalon.framework.configuration.Configuration configuration)
               throws org.apache.avalon.framework.configuration.ConfigurationException
Configures this object

Specified by:
configure in interface org.apache.avalon.framework.configuration.Configurable
Throws:
org.apache.avalon.framework.configuration.ConfigurationException

configureDatabase

protected void configureDatabase(org.apache.avalon.framework.configuration.Configuration configuration)
                          throws org.apache.avalon.framework.configuration.ConfigurationException
Configures the internal database

Throws:
org.apache.avalon.framework.configuration.ConfigurationException

configureHarvestIDGenerator

protected void configureHarvestIDGenerator(org.apache.avalon.framework.configuration.Configuration configuration)
                                    throws org.apache.avalon.framework.configuration.ConfigurationException
Configures the id generator for harvests

Throws:
org.apache.avalon.framework.configuration.ConfigurationException

getHarvesterId

protected java.lang.String getHarvesterId()
Returns an id for this harvester based upon the underlying document base id


configureAdminEmails

protected void configureAdminEmails(org.apache.avalon.framework.configuration.Configuration configuration)
                             throws org.apache.avalon.framework.configuration.ConfigurationException
Configures a list of admin emails can be sub-elements, a single attribute, or both

Parameters:
configuration -
Throws:
org.apache.avalon.framework.configuration.ConfigurationException

configureDataProviders

protected void configureDataProviders(org.apache.avalon.framework.configuration.Configuration configuration)
                               throws org.apache.avalon.framework.configuration.ConfigurationException
Configures data providers info that can be reused and from which requests can be automatically executed

Parameters:
configuration -
Throws:
org.apache.avalon.framework.configuration.ConfigurationException
See Also:
storedRequests

configureUpdateTriggers

protected void configureUpdateTriggers(java.lang.String requestUrl,
                                       org.apache.avalon.framework.configuration.Configuration updateConf)
                                throws org.apache.avalon.framework.configuration.ConfigurationException
Configures time triggers for stored requests

Parameters:
requestUrl - The request url
updateConf - The configuration for updates
Throws:
org.apache.avalon.framework.configuration.ConfigurationException
See Also:
#scheduler, #storedRequests

configureStoreRepositories

protected void configureStoreRepositories(java.lang.String repoUrl,
                                          org.apache.avalon.framework.configuration.Configuration oaiRepoConf)
                                   throws org.apache.avalon.framework.configuration.ConfigurationException
Configures the repositories to which data will be stored based upon their repository url

Parameters:
repoUrl - The repository/data provider url
oaiRepoConf - The configuration
Throws:
org.apache.avalon.framework.configuration.ConfigurationException

configurePipeline

protected void configurePipeline(org.apache.avalon.framework.configuration.Configuration configuration)
                          throws org.apache.avalon.framework.configuration.ConfigurationException
Configures the preIndexation pipeline

Parameters:
configuration -
Throws:
org.apache.avalon.framework.configuration.ConfigurationException
See Also:
pipe

getNewTempDir

protected java.io.File getNewTempDir()
                              throws SDXException,
                                     java.io.IOException
Creates a new temporary directory for writing harvested records before the will be indexed

Returns:
Throws:
SDXException
java.io.IOException

deleteTempDir

protected void deleteTempDir()
Deletes the directory represented by the tempDir class field


initTempDir

protected void initTempDir()
                    throws SDXException,
                           java.io.IOException
Establishes the tempDir class field

Throws:
SDXException
java.io.IOException

getIsoDate

protected java.lang.String getIsoDate()
Get's the current date in iso8601 format

Returns:

prepareRecordCapture

protected void prepareRecordCapture()
                             throws org.xml.sax.SAXException
Sets up resources to capture an oai record

Specified by:
prepareRecordCapture in class AbstractOAIHarvester
Throws:
org.xml.sax.SAXException

captureRecord

protected void captureRecord()
                      throws java.lang.Exception
Ends the capture of an oai record. Store properties (dateStamp and OAI identifier) needed for indexation.

Specified by:
captureRecord in class AbstractOAIHarvester
Throws:
java.lang.Exception

resetRecordCaptureFields

protected void resetRecordCaptureFields(boolean deleteDoc)
Resets the class fields for record capture possibility deleting the current harvetDoc object underlying file

Specified by:
resetRecordCaptureFields in class AbstractOAIHarvester
Parameters:
deleteDoc - flag for deletion of actual file

prepareRecordForDeletion

protected void prepareRecordForDeletion()
Sets up resources to delete an oai record Add the record to the list of the records to removed

Specified by:
prepareRecordForDeletion in class AbstractOAIHarvester

storeHarvestedData

protected boolean storeHarvestedData()
                              throws org.apache.cocoon.ProcessingException,
                                     java.io.IOException,
                                     SDXException,
                                     org.xml.sax.SAXException
Reads the documents from tempDir and indexes them in the corresponding document base, any marked deletions will be carried out as well

Specified by:
storeHarvestedData in class AbstractOAIHarvester
Returns:
Throws:
org.apache.cocoon.ProcessingException
SDXException
org.xml.sax.SAXException
java.io.IOException
See Also:
AbstractOAIHarvester.storeHarvestedData()

deleteOAIDocuments

protected void deleteOAIDocuments()
                           throws java.io.IOException,
                                  org.apache.cocoon.ProcessingException,
                                  SDXException,
                                  org.xml.sax.SAXException
Delete OAI documents from the current document base.

Throws:
java.io.IOException
org.apache.cocoon.ProcessingException
SDXException
org.xml.sax.SAXException

handleResumptionToken

protected void handleResumptionToken()
Handles the resumption token by issuing another request based upon the request from which the resumption token was received.

Specified by:
handleResumptionToken in class AbstractOAIHarvester

prepareResourceFromUrlIdentifierCapture

protected void prepareResourceFromUrlIdentifierCapture()
Prepares to read a url value from an oai record and retrieve the XML behind.

Specified by:
prepareResourceFromUrlIdentifierCapture in class AbstractOAIHarvester
See Also:
#identifierName, #currentMetadtaUrlIdentifier

captureResourceFromUrlIdentifier

protected void captureResourceFromUrlIdentifier()
Captures the xml from a url taken from an oai record and adds it to the oai-record as a sibling of the element

Specified by:
captureResourceFromUrlIdentifier in class AbstractOAIHarvester
See Also:
#currentMetadtaUrlIdentifier, #identifierName

resetAllFields

protected void resetAllFields()
Resets necessary class fields

Overrides:
resetAllFields in class AbstractOAIHarvester

getIndexParameters

protected IndexParameters getIndexParameters()
Builds simple index parameters for indexation of oai records into the undelryi

Returns:

sendStoredHarvestingRequests

public void sendStoredHarvestingRequests()
                                  throws org.xml.sax.SAXException
Sends the details of stored harvesting requests to the current consumer

Specified by:
sendStoredHarvestingRequests in interface OAIHarvester
Throws:
org.xml.sax.SAXException

targetTriggered

public void targetTriggered(java.lang.String triggerName)
Triggers a oai request to a repository based upon a trigger name (also a request url)

Specified by:
targetTriggered in interface Target
Parameters:
triggerName -

startElement

public void startElement(java.lang.String s,
                         java.lang.String s1,
                         java.lang.String s2,
                         org.xml.sax.Attributes attributes)
                  throws org.xml.sax.SAXException
Description copied from class: AbstractSynchronizedXMLPipe
Receive notification of the beginning of an element.

Specified by:
startElement in interface org.xml.sax.ContentHandler
Overrides:
startElement in class AbstractOAIHarvester
Throws:
org.xml.sax.SAXException

endElement

public void endElement(java.lang.String s,
                       java.lang.String s1,
                       java.lang.String s2)
                throws org.xml.sax.SAXException
Description copied from class: AbstractSynchronizedXMLPipe
Receive notification of the end of an element.

Specified by:
endElement in interface org.xml.sax.ContentHandler
Overrides:
endElement in class AbstractOAIHarvester
Throws:
org.xml.sax.SAXException

shouldHarvestDocument

protected boolean shouldHarvestDocument()
Querys the underlying data structures based upon current sax flow position/set class fields and determines whether an oai record should be harvested

Specified by:
shouldHarvestDocument in class AbstractOAIHarvester
Returns:
boolean indicates whether the record should be handled

saveCriticalFields

protected void saveCriticalFields(boolean dataHarvested)
                           throws org.xml.sax.SAXException
Saves critical data about a harvest

Specified by:
saveCriticalFields in class AbstractOAIHarvester
Parameters:
dataHarvested -
Throws:
org.xml.sax.SAXException

generateNewHarvestId

protected java.lang.String generateNewHarvestId()
Generates an id to associate with a harvest

Returns:

sendPastHarvestsSummary

public void sendPastHarvestsSummary()
                             throws org.xml.sax.SAXException
Sends sax events to the current consumer with summary details of the all the past harvests

Specified by:
sendPastHarvestsSummary in interface OAIHarvester
Throws:
org.xml.sax.SAXException

lastUpdated

public java.util.Date lastUpdated()
Retrieves the time when the harvester was last updated

Returns:

purgePastHarvestsData

public void purgePastHarvestsData()
Destroys all summary data pertaining to past harvests but not the actual oai records harvested

Specified by:
purgePastHarvestsData in interface OAIHarvester

storeFailedHarvestData

protected void storeFailedHarvestData(java.lang.Exception e)
Stores data about harvesting failures caused by problems other than oai errors sent from a queried repository

Specified by:
storeFailedHarvestData in class AbstractOAIHarvester
Parameters:
e -


Copyright © 2000-2004 Ministere de la culture et de la communication / AJLSM. All Rights Reserved.