dl.queryClient module

querymanager.queryClient

Client methods for the DataLab Query Manager Service.

Query Manager Client Interface:

            isAlive  (svc_url=DEF_SERVICE_URL, timeout=2)
        set_svc_url  (svc_url)
        get_svc_url  ()
        set_profile  (profile)
        get_profile  ()
      list_profiles  (optval, token=None, profile=None, format='text')
      list_profiles  (token=None, profile=None, format='text')

set_timeout_request  (nsec)
get_timeout_request  ()
             schema  (value, format='text', profile=None)
             schema  (value='', format='text', profile=None)
           services  (name=None, svc_type=None, format=None,
                      profile='default')

              query  (token, query, adql=None, sql=None, fmt='csv', out=None,
                      async_=False, profile='default', **kw)
              query  (optval, adql=None, sql=None, fmt='csv', out=None,
                      async_=False, profile='default', **kw)
              query  (token=None, adql=None, sql=None, fmt='csv', out=None,
                      async_=False, profile='default', **kw)
             status  (token, jobId, profile='default')
             status  (optval, jobId=None, profile='default')
             status  (token=None, jobId=None, profile='default')
               jobs  (token, jobId, status='all', option='list')
               jobs  (optval, jobId=None, status='all', option='list')
               jobs  (token=None, jobId=None, status='all', option='list')
            results  (token, jobId, delete=True, profile='default')
            results  (optval, jobId=None, delete=True, profile='default')
            results  (token=None, jobId=None, delete=True, profile='default')
              error  (token, jobId, profile='default')
              error  (optval, jobId=None, profile='default')
              error  (token=None, jobId=None, profile='default')
              abort  (token, jobId, profile='default')
              abort  (optval, jobId=None, profile='default')
              abort  (token=None, jobId=None, profile='default')
               wait  (token, jobId, wait=3, verbose=False,
                      profile='default')
               wait  (optval, jobId=None, wait=3, verbose=False,
                      profile='default')
               wait  (token=None, jobId=None, wait=3, verbose=False,
                      profile='default')

          mydb_list  (optval, table=None, **kw)
          mydb_list  (table=None, token=None, **kw)
        mydb_create  (token, table, schema, **kw)
        mydb_create  (table, schema, token=None, **kw)
        mydb_insert  (token, table, data, **kw)
        mydb_insert  (table, data, token=None, **kw)
        mydb_import  (token, table, data, **kw)
        mydb_import  (table, data, token=None, **kw)
      mydb_truncate  (token, table)
      mydb_truncate  (table, token=None)
         mydb_index  (token, table, column)
         mydb_index  (table, column, token=None)
          mydb_drop  (token, table)
          mydb_drop  (table, token=None)
        mydb_rename  (token, source, target)
        mydb_rename  (source, target, token=None)
          mydb_copy  (token, source, target)
          mydb_copy  (source, target, token=None)

               list  (token, table)                  # DEPRECATED
               list  (optval, table=None)            # DEPRECATED
               list  (table=None, token=None)        # DEPRECATED
               drop  (token, table)                  # DEPRECATED
               drop  (optval, table=None)            # DEPRECATED
               drop  (table=None, token=None)        # DEPRECATED

Import via:

from dl import queryClient
exception dl.queryClient.DLHTTPException(status_code, message)[source]

Bases: Exception

dl.queryClient.abort()[source]

Abort the specified Asynchronous job.

Usage:

abort (token=None, jobId=None)

MultiMethod Usage:

queryClient.abort (token, jobId)
queryClient.abort (jobId)
queryClient.abort (token, jobId=<id>)
queryClient.abort (jobId=<str>)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • jobId (str) – The jobId to abort.

Returns:

results

Return type:

str

Example

# issue an Async query (here a tiny one just for this example)
query = 'select ra,dec from gaia_dr3.gaia_source limit 3'
jobId = queryClient.query(adql=query, fmt='csv', async_=True)

# ensure job completes...then check status and retrieve results
time.sleep(4)
if queryClient.status(jobId) == 'COMPLETED':
    results = queryClient.results(jobId)
    print (type(results))
    print (results)

This prints

<class 'str'>
ra,dec
314.14738713961134,37.33643208345386
314.1588238386895,37.33682543157976
314.1519366421579,37.33533842266872
dl.queryClient.drop()[source]

[DEPRECATED] Drop the specified table from the user’s MyDB.

Usage:

drop (table=None, token=None)

MultiMethod Usage:

queryClient.drop (token, table)
queryClient.drop (table)
queryClient.drop (token, table=<id>)
queryClient.drop (table=<str>)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • table (str) – The specific table to drop.

dl.queryClient.error()[source]

Retrieve the error of an Asynchronous query, once completed.

Usage:

error (token=None, jobId=None)

MultiMethod Usage:

queryClient.error (jobId)
queryClient.error (token, jobId)
queryClient.error (token, jobId=<id>)
queryClient.error (jobId=<str>)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • jobId (str) – The jobId returned when issuing an Asynchronous query via queryClient.query() with async_=True.

Returns:

error

Return type:

str

Example

# issue an Async query (here a tiny one just for this example)
query = 'select ra,dec1 from gaia_dr3.gaia_source limit 3'
jobId = queryClient.query(adql=query, fmt='csv', async_=True)

# ensure job completes...then check status and retrieve error
time.sleep(4)
if queryClient.status(jobId) == 'ERROR':
    error = queryClient.error(jobId)
    print (type(error))
    print (error)

This prints

<class 'str'>
Error: IllegalArgumentException: Column [dec1] does not exist.
dl.queryClient.getClient(profile='default', svc_url='https://datalab.noirlab.edu/query')[source]

Create a new queryClient object and set a default profile.

dl.queryClient.get_profile()[source]

Get the current query profile.

Parameters:

None

Returns:

profile – The name of the current profile used with the Query Manager service.

Return type:

str

Example

print ("Query Service profile = " + queryClient.get_profile())
dl.queryClient.get_svc_url()[source]

Get the currently-used Query Manager serice URL.

Parameters:

None

Returns:

service_url – The currently-used Query Service URL.

Return type:

str

Example

print (queryClient.get_svc_url())
dl.queryClient.get_timeout_request()[source]

Get the current Sync query timeout value.

Parameters:

None

Returns:

result – Current Sync query timeout value in seconds.

Return type:

int

Example

# get the current timeout value
print (queryClient.get_timeout_request())
dl.queryClient.human_readable_size(size, decimal_places=2)[source]

Converts a size in bytes to a human-readable format (e.g., KB, MB, GB).

Parameters:

  • size – Size in bytes.

  • decimal_places – Number of decimal places to format.

Returns:

Human-readable size string.

dl.queryClient.human_readable_time(seconds, decimal_places=2)[source]

Converts time in seconds to a human-readable format (hours:minutes:seconds).

Parameters:

  • seconds – Time in seconds.

  • decimal_places – Number of decimal places for the seconds part.

Returns:

Human-readable time string.

dl.queryClient.isAlive(svc_url='https://datalab.noirlab.edu/query', timeout=5)[source]
Check whether the QueryManager service at the given URL is

alive and responding. This is a simple call to the root service URL or ping() method.

Parameters:

  • svc_url (str) – The Query Service URL to ping.

  • timeout (int) – Call will assume to have failed if timeout seconds pass.

Returns:

result – True if service responds properly, False otherwise.

Return type:

bool

Example

if queryClient.isAlive():
    print ("Query Manager is alive")
dl.queryClient.jobs()[source]

Get a list of the user’s Async jobs.

Usage:

jobs (token=None, jobId=None, format='text', status='all')

MultiMethod Usage:

queryClient.jobs (jobId)
queryClient.jobs (token, jobId)
queryClient.jobs (token, jobId=<id>)
queryClient.jobs (jobId=<str>)

Use the authentication token and the jobId of a previously issued Asynchronous query to check the query’s current status.

Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • jobId (str) – The jobId returned when issuing an Asynchronous query via queryClient.query() with async_=True.

  • format (str) – Format of the result. Support values include ‘text’ for a simple formatted table suitable for printing, or ‘json’ for a JSON string of the full matching record(s).

  • status (str) –

    If status='all' then all Async jobs are returned, otherwise this value may be used to return only those jobs with the specified status. Allowed values are:

    • ’all’ Return all jobs.

    • ’EXECUTING’ Job is still running.

    • ’COMPLETED’ Job completed successfully.

    • ’ERROR’ Job exited with an error.

    • ’ABORTED’ Job was aborted by the user.

  • option (str) – If list then the matching records are returned, if delete then the records are removed from the database (e.g. to clear up long job lists of completed jobs).

Returns:

joblist – Returns a list of Async query jobs submitted by the user in the last 30 days, possibly filtered by the status parameter. The ‘json’ format option allows the caller to format the full contents of the job record beyond the supplied simple ‘text’ option.

Return type:

str

Example

print (queryClient.jobs(jobId))

This prints

JobID              Start              End                 Status
tfu8zpn2tkrlfyr9e  07-22-20T13:10:22  07-22-20T13:34:12   COMPLETED
k8uznptrkkl29ryef  07-22-20T14:09:45                      EXECUTING
      :                   :                 :                :
dl.queryClient.list()[source]

List the tables or table schema in the user’s MyDB.

Usage:

list (table=None, token=None)

MultiMethod Usage:

queryClient.list (token, table)
queryClient.list (table)
queryClient.list (token, table=<id>)
queryClient.list ()
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • table (str) – The specific table to list (returns the table schema), or an empty string to return a list of the names of all tables.

Returns:

listing – The list of tables in the user’s MyDB or the schema of the named table.

Return type:

str

Example

# List the tables
queryClient.list()
dl.queryClient.list_profiles()[source]

Retrieve the profiles supported by the query manager service.

Usage:

list_profiles (token=None, profile=None, format='text')

MultiMethod Usage:

queryClient.list_profiles (token)
queryClient.list_profiles ()
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • profile (str) – A specific profile configuration to list. If None, a list of profiles available to the given auth token is returned.

  • format (str) – Result format: One of ‘text’ or ‘json’.

Returns:

profiles – A list of the names of the supported profiles or a dictionary of the specific profile.

Return type:

list or dict

Example

profiles = queryClient.list_profiles()
profiles = queryClient.list_profiles(token)
dl.queryClient.mydb_copy()[source]

Copy a table in the user’s MyDB to a new name.

Usage:

mydb_copy (source, target, token=None)

MultiMethod Usage:

queryClient.mydb_copy (token, source, target)
queryClient.mydb_copy (source, target)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • source (str) – The old table name, i.e. the table to be copied.

  • target (str) – The new table name, i.e. the table to be created.

Return type:

command status

Example

# Copy table 'foo' to a new table, 'bar'
queryClient.mydb_copy('foo', 'bar')
dl.queryClient.mydb_create()[source]

Create a table in the user’s MyDB.

Usage:

mydb_create (table, schema, token=None, **kw)

MultiMethod Usage:

queryClient.mydb_create (token, table, <schema_dict>)
queryClient.mydb_create (table, <schema_dict>)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • table (str) – The name of the table to create.

  • schema (str or dict) – The schema is CSV text containing the name of the column and it’s PostgreSQL data type. If set as a ‘str’ type it is either a CSV string, or the name of a file containing the CSV. If passed as a ‘dict’ type, it is a dictionary object where keys are the column names and values are the data types.

  • drop (bool [Optional]) – Drop any existing table of the same name before creating new one.

Example

# Create table in MyDB named 'foo' with columns defined by
# the file 'schema.txt'.  The schema file contains:
#
#     id,text
#     ra,double precision
#     dec,double precision
#
queryClient.mydb_create ('foo', 'schema.txt')
dl.queryClient.mydb_drop()[source]

Drop the specified table from the user’s MyDB.

Usage:

mydb_drop (table, token=None)

MultiMethod Usage:

queryClient.mydb_drop (token, table)
queryClient.mydb_drop (table)
queryClient.mydb_drop (token, table=<id>)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • table (str) – The specific table to drop.

Return type:

command status

Example

# Drop the 'foo1' table
queryClient.mydb_drop('foo1')
dl.queryClient.mydb_flush(token=None)[source]
dl.queryClient.mydb_import()[source]

Import data into a table in the user’s MyDB.

Usage:

mydb_import (table, data, token=None, **kw)

MultiMethod Usage:

queryClient.mydb_import (token, table, data)
queryClient.mydb_import (table, data)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • table (str) – The name of the table to be loaded.

  • data (str or data object) –

    The data file or python object to be loaded. The data value may be one of the following types:

    • filename – A CSV file of data.

    • string – A string containing CSV data.

    • Pandas DataFrame – A Pandas DataFrame object.

    Additional object types can be added provided the data can be converted to a CSV format.

  • schema (str [Optional]) – If set, this is a filename or string containing a schema for the data table to be created. A schema contains a comma-delimited row for each column containing the column name and it’s Postgres data type. If not set, the schema is determined automatically from the data.

  • append (bool [Optional]) – Append any existing table of the same name.

  • verbose (bool [Optional]) – Be verbose about operations.

Returns:

  • schema (str) – A string containing the table schema.

  • data_obj (str or data object) – The CSV data to be imported (possibly converted).

Example

# Import data into a MyDB table named 'foo' from file 'data.csv'.
schema, data = queryClient.mydb_import ('foo', 'data.csv')
dl.queryClient.mydb_index()[source]

Index the specified column in a table in the user’s MyDB.

MultiMethod Usage:

queryClient.mydb_index (table, colunm)
queryClient.mydb_index (token, table, column)
queryClient.mydb_index (table, column, token=None)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • table (str) – The table containing the column to be indexed.

  • column (str) – The column to be indexed.

  • q3c (str) – A comma-delimited list of two column names giving the RA and Dec positions (decimal degrees) to be used to Q3C index the table. If None, no Q3C index will be computed.

  • cluster (bool) – If enabled, data table will be rewritten to cluster on the Q3C index for efficiency. Only used when ‘q3c’ columns are specified.

  • async (bool) – If enabled, index commands will be submitted Asynchronously.

Return type:

command status

Example
# In the ‘foo1’ table, index the ‘id’ column

queryClient.mydb_index(‘foo1’, ‘id’)

# Index and cluster the table by position

queryClient.mydb_index(‘foo1’, q3c=’ra,dec’, cluster=True)

dl.queryClient.mydb_insert()[source]

Insert data into a table in the user’s MyDB.

Usage:

mydb_insert (table, data, token=None, **kw)

MultiMethod Usage:

queryClient.mydb_insert (token, table, <filename>)
queryClient.mydb_insert (token, table, <data_object>)
queryClient.mydb_insert (table, <filename>)
queryClient.mydb_insert (table, <data_object>)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • table (str) – The name of the table to append.

  • data (str or data object) – The schema is CSV text containing the name of the column and it’s PostgreSQL data type. If set as a ‘str’ type it is either a CSV string, or the name of a file containing the CSV data. If passed as a tabular data object, it is converted to CSV and sent to the service.

  • csv_header (bool [Optional]) – If True, then the CSV data object contains a CSV header line, i.e. the first line is a row of column names. Otherwise, no column names are assumed and the column order must match the table schema.

Example

# Insert data from a CSV file called 'data.csv' into a MyDB table
# named 'foo'.
queryClient.mydb_insert ('foo', 'data.csv')
dl.queryClient.mydb_list()[source]

List the tables or table schema in the user’s MyDB.

Usage:

mydb_list (table=None, token=None, **kw)

MultiMethod Usage:

queryClient.mydb_list (table)
queryClient.mydb_list (token, table=<str>)
queryClient.mydb_list (table=<str>)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • table (str) – The specific table to list (returns the table schema), or an empty string to return a list of the names of all tables.

Returns:

listing – The list of tables in the user’s MyDB or the schema of the named table.

Return type:

str

Example

# List the tables
queryClient.mydb_list()
dl.queryClient.mydb_rename()[source]

Rename a table in the user’s MyDB to a new name.

Usage:

mydb_rename (source, target, token=None)

MultiMethod Usage:

queryClient.mydb_rename (token, source, target)
queryClient.mydb_rename (source, target)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • source (str) – The old table name.

  • target (str) – The new table name.

Return type:

command status

Example

# Rename table 'foo' to a new name, 'bar'
queryClient.mydb_rename('foo', 'bar')
dl.queryClient.mydb_truncate()[source]

Truncate the specified table in the user’s MyDB.

Usage:

mydb_truncate (table, token=None)

MultiMethod Usage:

queryClient.mydb_truncate (token, table)
queryClient.mydb_truncate (table)
queryClient.mydb_truncate (token, table=<id>)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • table (str) – The specific table to truncate.

Example

# Truncate the table 'foo'
queryClient.mydb_truncate('foo')
dl.queryClient.progress_report(download_t, download_d, upload_t, upload_d)[source]

Callback function to report the progress of a download or upload operation.

This function is designed to be used with pycurl to periodically report the progress of file transfer operations. It prints updates to the console at intervals specified by the progress.report_interval.

Parameters: - download_t (int): Total size of the download in bytes. - download_d (int): Number of bytes downloaded so far. - upload_t (int): Total size of the upload in bytes (not used in this context). - upload_d (int): Number of bytes uploaded so far (not used in this context).

dl.queryClient.qcToString(s)[source]

qcToString – Force a return value to be type ‘string’ for all Python versions.

dl.queryClient.query()[source]

Send an SQL or ADQL query to the database or TAP service.

Usage:

query (token=None, adql=None, sql=None, fmt='csv', out=None,
       async_=False, drop=False, profile='default', **kw):

MultiMethod Usage:

queryClient.query (token, query, <args>)
queryClient.query (token | query, <args>)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • adql (str or None) –

    ADQL query string that will be passed to the DB query manager, e.g.

    adql='select top 3 ra,dec from gaia_dr3.gaia_source'

  • sql (str or None) –

    SQL query string that will be passed to the DB query manager, e.g.

    sql='select ra,dec from gaia_dr3.gaia_source limit 3'

  • fmt (str) –

    Format of result to be returned by the query. Permitted values are:

    • ’csv’ The returned result is a comma-separated string

      that looks like a csv file (newlines at the end of every row) [DEFAULT].

    • ’csv-noheader’ A csv result with no column headers (data only).

    • ’ascii’ Same, but the column separator is a tab .

    • ’array’ Returns a NumPy array.

    • ’pandas’ Returns a Pandas DataFrame.

    • ’structarray’ Numpy structured array (aka ‘record array’).

    • ’table’ Returns an Astropy Table object.

      The following formats may be used when saving a file to virtual storage on the server:

    • ’fits’ FITS binary.

    • ’votable’ An XML-formatted VOTable.

  • out (str or None) – The output filename to create on the local machine, the URI of a VOSpace or MyDB resource to create, or None if the result is to be returned directly to the caller.

  • async (bool) –

    If True, the query is Asynchronous, i.e. a job is submitted to the DB, and a jobID token is returned the caller. The jobID must be then used to check the query’s status and to retrieve the result (when the job status is COMPLETED) or the error message (when the job status is ERROR). Default is False, i.e. the task runs a Synchroneous query.

    async_ replaces the previous async parameter, because async was promoted to a keyword in Python 3.7. Users of Python versions prior to 3.7 can continue to use the async keyword.

  • drop (bool) – If True, then if the query is saving to MyDB where the same table name already exists, it will overwrite the old MyDB table.

  • profile (str or None) – The Query Manager profile to use for this call. If None then the default profile is used. Available profiles may be listed using the queryClient.list_profiles().

  • **kw (dict) –

    Optional keyword arguments. Supported keywords currently include:

    wait = False

    Wait for Asynchronous queries to complete? If enabled, the query() method will submit the job in Async mode and then poll for results internally before returning. The default is to return the job ID immediately and let the client poll for job status and return results.

    timeout = 300

    Requested timeout (in seconds) for a query. For a Sync query, this value sets a session timeout request in the database that will abort the query at the specified time. A maximum value of 600 seconds is permitted. If the wait option is enabled for an Async query, this is the maximum time the query will be allowed to run before an abort() is issued on the job. The maximum timeout for an Async job is 24-hrs (86400 sec).

    poll = 1

    Async job polling time in seconds.

    verbose = False

    Print verbose messages during Async job.

Returns:

result – If async_=False, the return value is the result of the query as a formatted string (see fmt). Otherwise the result string is a job token, with which later the Asynchronous query’s status can be checked (queryClient.status()), and the result retrieved (see queryClient.result()).

Return type:

str

Example

query = 'select ra,dec from gaia_dr3.gaia_source limit 3'
response = queryClient.query(adql=query, fmt='csv')
print (response)

This prints

ra,dec
314.14738713961134,37.33643208345386
314.1588238386895,37.33682543157976
314.1519366421579,37.33533842266872
class dl.queryClient.queryClient(profile='default', svc_url='https://datalab.noirlab.edu/query')[source]

Bases: object

QUERYCLIENT – Client-side methods to access the Data Lab

Query Manager Service.

abort(token=None, jobId=None, profile='default')[source]

Usage:: queryClient.abort (jobID=<str>)

chunked_upload(token, local_file, remote_file)[source]

A streaming file uploader.

conequery(token, input=None, out=None, schema=None, table=None, ra=None, dec=None, search=0.5)[source]

Send a cone search query to the consearch service

dataType(val, current_type)[source]

Lexically scan a value to determine the datatype.

drop(token=None, table=None)[source]

Usage:: queryClient.drop (table=<str>)

error(token=None, jobId=None, profile='default')[source]

Usage:: queryClient.error (jobID=<str>)

getFromURL(svc_url, path, token)[source]

Get something from a URL. Return a ‘response’ object

getHeaders(token)[source]

Get default tracking headers,

getSchema(data, **kw)[source]

Generate a schema for mydb_create() from a CSV file or data object.

getStreamURL(url, headers, fname=None, chunk_size=1048576, progress=False)[source]
get_profile()[source]

Get the current query profile.

Parameters:

None

Returns:

profile – The name of the current profile used with the Query Manager service.

Return type:

str

Example

print ("Query Service profile = " + queryClient.get_profile())
get_svc_url()[source]

Get the currently-used Query Manager serice URL.

Parameters:

None

Returns:

service_url – The currently-used Query Service URL.

Return type:

str

Example

print (queryClient.get_svc_url())
get_timeout_request()[source]

Get the current Sync query timeout value.

Parameters:

None

Returns:

result – Current Sync query timeout value in seconds.

Return type:

int

Example

# get the current timeout value
print (queryClient.get_timeout_request())
isAlive(svc_url=None, timeout=5)[source]
Check whether the QueryManager service at the given URL is

alive and responding. This is a simple call to the root service URL or ping() method.

Parameters:

  • svc_url (str) – The Query Service URL to ping.

  • timeout (int) – Call will assume to have failed if timeout seconds pass.

Returns:

result – True if service responds properly, False otherwise.

Return type:

bool

Example

if queryClient.isAlive():
    print ("Query Manager is alive")
jobs(token=None, jobId=None, format='text', status='all', option='list')[source]

Usage: queryClient.jobs (jobID=<str>)

list(token=None, table=None)[source]

Usage:: queryClient.list (table=<str>)

list_profiles(token=None, profile=None, format='text')[source]

Usage: queryClient.client.list_profiles (…)

mydb_copy(**kw)[source]

Call the appropriate instance of the function.

mydb_create(**kw)[source]

Call the appropriate instance of the function.

mydb_drop(**kw)[source]

Call the appropriate instance of the function.

mydb_import(**kw)[source]

Call the appropriate instance of the function.

mydb_index(**kw)[source]

Call the appropriate instance of the function.

mydb_insert(**kw)[source]

Call the appropriate instance of the function.

mydb_list(token=None, table=None, index=False, **kw)[source]

Usage:: queryClient.mydb_list (table=<str>)

mydb_rename(**kw)[source]

Call the appropriate instance of the function.

mydb_truncate(**kw)[source]

Call the appropriate instance of the function.

static pretty_print_POST(req)[source]

At this point it is completely built and ready to be fired; it is “prepared”.

However pay attention at the formatting used in this function because it is programmed to be pretty printed and may differ from the actual request.

pycurl_download(url, headers, fname=None, progress=False)[source]
query(token=None, adql=None, sql=None, fmt='csv', out=None, async_=False, drop=False, profile='default', **kw)[source]

Usage: queryClient.client.query (…)

request_download(url, headers, fname=None, chunk_size=1048576)[source]

Get the specified URL in a streaming fashion. This allows for large downloads without hitting timeout limits.

results(token=None, jobId=None, fname=None, delete=True, profile='default', progress=False)[source]

Usage:: queryClient.results (jobID=<str>)

schema(value='', format='text', profile=None)[source]

Usage: queryClient.schema ([value])

services(name=None, svc_type=None, mode='list', profile='default')[source]

Usage: queryClient.services ()

set_profile(profile)[source]

Set the service profile to be used.

Parameters:

profile (str) – The name of the profile to use. The list of available profiles can be retrieved from the service (see function queryClient.list_profiles()).

Return type:

Nothing

Example

queryClient.set_profile('test')
set_svc_url(svc_url)[source]

Set the Query Manager service URL.

Parameters:

svc_url (str) – The service URL of the Query Manager to call.

Return type:

Nothing

Example

queryClient.set_svc_url("http://localhost:7002")
set_timeout_request(nsec)[source]

Set the requested Sync query timeout value (in seconds).

Parameters:

nsec (int) – The number of seconds requested before a Sync query timeout occurs. The service may cap this as a server defined maximum.

Return type:

Nothing

Example

# set the Sync query timeout request to 30 seconds
queryClient.set_timeout_request(30)
siaquery(token, input=None, out=None, search=0.5)[source]

Send a SIA (Simple Image Access) query to the query manager service

status(token=None, jobId=None, profile='default')[source]

Usage:: queryClient.status (jobID=<str>)

wait(token=None, jobId=None, wait=3, verbose=False, profile='default')[source]

Usage:: queryClient.wait (jobID=<str>)

exception dl.queryClient.queryClientError(message)[source]

Bases: Exception

dl.queryClient.removeComment(s)[source]
dl.queryClient.results()[source]

Retrieve the results of an Asynchronous query, once completed.

Usage:

results (token=None, jobId=None, delete=True)

MultiMethod Usage:

queryClient.results (jobId)
queryClient.results (token, jobId)
queryClient.results (token, jobId=<id>)
queryClient.results (jobId=<str>)
Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • jobId (str) – The jobId returned when issuing an Asynchronous query via queryClient.query() with async_=True.

  • progress (bool) – Set to False by default. If progress set to True it will report the download progress.

Returns:

results

Return type:

str

Example

# issue an Async query (here a tiny one just for this example)
query = 'select ra,dec from gaia_dr3.gaia_source limit 3'
jobId = queryClient.query(adql=query, fmt='csv', async_=True)

# ensure job completes...then check status and retrieve results
time.sleep(4)
if queryClient.status(jobId) == 'COMPLETED':
    results = queryClient.results(jobId)
    print (type(results))
    print (results)

This prints

<type 'str'>
ra,dec
314.14738713961134,37.33643208345386
314.1588238386895,37.33682543157976
314.1519366421579,37.33533842266872
dl.queryClient.schema()[source]

[DEPRECATED] Return information about a data service schema.

Usage:

schema (value='', format='text', profile=None)
Parameters:

  • value (str) – Schema object to return: Of the form <schema>[.<table>[.<column]].

  • profile (str) – The name of the service profile to use. The list of available profiles can be retrieved from the service (see function queryClient.list_profiles()).

  • format (str) – Result format: One of ‘text’ or ‘json’ (NOT CURRENTLY USED).

Return type:

Anything?

dl.queryClient.services(name=None, svc_type=None, mode='list', profile='default')[source]

Usage: queryClient.services ()

dl.queryClient.set_profile(profile)[source]

Set the service profile to be used.

Parameters:

profile (str) – The name of the profile to use. The list of available profiles can be retrieved from the service (see function queryClient.list_profiles()).

Return type:

Nothing

Example

queryClient.set_profile('test')
dl.queryClient.set_svc_url(svc_url)[source]

Set the Query Manager service URL.

Parameters:

svc_url (str) – The service URL of the Query Manager to call.

Return type:

Nothing

Example

queryClient.set_svc_url("http://localhost:7002")
dl.queryClient.set_timeout_request(nsec)[source]

Set the requested Sync query timeout value (in seconds).

Parameters:

nsec (int) – The number of seconds requested before a Sync query timeout occurs. The service may cap this as a server defined maximum.

Return type:

Nothing

Example

# set the Sync query timeout request to 30 seconds
queryClient.set_timeout_request(30)
dl.queryClient.setup_progress()[source]

Setup the progress reporting.

dl.queryClient.status()[source]

Get the status of an Asynchronous query.

Usage:

status (token=None, jobId=None)

MultiMethod Usage:

queryClient.status (jobId)
queryClient.status (token, jobId)
queryClient.status (token, jobId=<id>)
queryClient.status (jobId=<str>)

Use the authentication token and the jobId of a previously issued Asynchronous query to check the query’s current status.

Parameters:

  • token (str [Optional]) – Authentication token (see function authClient.login()).

  • jobId (str) – The jobId returned when issuing an Asynchronous query via queryClient.query() with async_=True.

Returns:

status – Either QUEUED or EXECUTING or COMPLETED. If the token & jobId combination does not correspond to an actual job, then a HTML-formatted error message is returned. If there is a problem with the backend, the returned value can be ERROR.

When status is COMPLETED, you can retrieve the results of the query via queryClient.results().

Return type:

str

Example

import time
query = 'select ra,dec from gaia_dr3.gaia_source limit 200000'
jobId = queryClient.query(adql=query, fmt='csv', async_=True)
while True:
    status = queryClient.status(jobId)
    print ("time index =", time.localtime()[5], "   status =", status)
    if status == 'COMPLETED':
        break
    time.sleep(1)

This prints

time index = 16    status = EXECUTING
time index = 17    status = EXECUTING
time index = 18    status = COMPLETED
dl.queryClient.wait()[source]

Loop until an Async job has completed.

Usage:

queryClient.wait (jobID=<str>)
Parameters:

  • jobId (str) – The job ID string of a submitted query job.

  • wait (int or float) – Wait for wait seconds before checking status again. Default: 3sec.