depot.py revision 1937
20N/A# The contents of this file are subject to the terms of the 20N/A# Common Development and Distribution License (the "License"). 20N/A# You may not use this file except in compliance with the License. 20N/A# See the License for the specific language governing permissions 20N/A# and limitations under the License. 20N/A# When distributing Covered Code, include this CDDL HEADER in each 20N/A# If applicable, add the following below this CDDL HEADER, with the 20N/A# fields enclosed by brackets "[]" replaced with your own identifying 20N/A# information: Portions Copyright [yyyy] [name of copyright owner] 1902N/A# Copyright (c) 2007, 2010 Oracle and/or its affiliates. All rights reserved. 22N/A# pkg.depotd - package repository daemon 50N/A# XXX The prototype pkg.depotd combines both the version management server that 50N/A# answers to pkgsend(1) sessions and the HTTP file server that answers to the 50N/A# various GET operations that a pkg(1) client makes. This split is expected to 50N/A# be made more explicit, by constraining the pkg(1) operations such that they 50N/A# can be served as a typical HTTP/HTTPS session. Thus, pkg.depotd will reduce 50N/A# to a special purpose HTTP/HTTPS server explicitly for the version management 50N/A# operations, and must manipulate the various state files--catalogs, in 50N/A# particular--such that the pkg(1) pull client can operately accurately with 50N/A# XXX Although we pushed the evaluation of next-version, etc. to the pull 50N/A# client, we should probably provide a query API to do same on the server, for 50N/A# dumb clients (like a notification service). 589N/A# The default path for static and other web content. 965N/A# cherrypy has a max_request_body_size parameter that determines whether the 965N/A# server should abort requests with REQUEST_ENTITY_TOO_LARGE when the request 965N/A# body is larger than the specified size (in bytes). The maximum size supported 965N/A# by cherrypy is 2048 * 1024 * 1024 - 1 (just short of 2048MB), but the default 965N/A# here is purposefully conservative. 382N/A# The minimum number of threads allowed. 382N/A# The default number of threads to start. 382N/A# The maximum number of threads that can be started. 382N/A# The default server socket timeout in seconds. We want this to be longer than 382N/A# the normal default of 10 seconds to accommodate clients with poor quality 382N/A# Whether modify operations should be allowed. 382N/A# Whether the repository catalog should be rebuilt on startup. 429N/A# Whether the indexes should be rebuilt 461N/A# Not in mirror mode by default 1902N/A# Not in link-local mirror mode my default 751N/A print >>
sys.
stderr,
"""cherrypy 3.1.0 or greater (but less than """ \
751N/A """3.2.0) is required to use this program.""" 466N/A """This is a dummy object that we can use to discard log entries 466N/A without relying on non-portable interfaces such as /dev/null.""" 466N/A """Discard the bits.""" 466N/A """Discard the bits.""" 1633N/A """Optionally emit a usage message and then exit using the specified 1633N/A # The full usage message isn't desired. 1633N/A emsg(_(
"Try `pkg.depotd --help or -?' for more " 1431N/A [-t socket_timeout] [--cfg-file] [--content-root] 1633N/A [--disable-ops op[/1][,...]] [--debug feature_list] 1902N/A [--file-root dir] [--log-access dest] [--log-errors dest] 1902N/A [--mirror] [--nasty] [--set-property <section.property>=<value>] 1937N/A [--proxy-base url] [--readonly] [--rebuild] [--socket-path] 1937N/A [--ssl-cert-file] [--ssl-dialog] [--ssl-key-file] 1937N/A [--sort-file-max-size size] [--writable-root dir] 1542N/A --add-content Check the repository on startup and add any new 1542N/A packages found. Cannot be used with --mirror or 812N/A --cfg-file The pathname of the file from which to read and to 812N/A write configuration information. 589N/A --content-root The file system path to the directory containing the 589N/A the static and other web content used by the depot's 589N/A browser user interface. The default value is 1431N/A --disable-ops A comma separated list of operations that the depot 1431N/A should not configure. If, for example, you wanted 1431N/A to omit loading search v1, 'search/1' should be 1431N/A provided as an argument, or to disable all search 1431N/A operations, simply 'search'. 858N/A --debug The name of a debug feature to enable; or a whitespace 1633N/A or comma separated list of features to enable. 1633N/A Possible values are: headers. 1633N/A --exit-ready Perform startup processing (including rebuilding 1633N/A catalog or indices, if requested) and exit when 1633N/A ready to start serving packages. 1902N/A --file-root The path to the root of the file content for a given 1902N/A repository. This is used to override the default, 466N/A --log-access The destination for any access related information 466N/A logged by the depot process. Possible values are: 466N/A stderr, stdout, none, or an absolute pathname. The 466N/A default value is stdout if stdout is a tty; otherwise 466N/A the default value is none. 466N/A --log-errors The destination for any errors or other information 466N/A logged by the depot process. Possible values are: 466N/A stderr, stdout, none, or an absolute pathname. The 466N/A default value is stderr. 589N/A --mirror Package mirror mode; publishing and metadata operations 589N/A disallowed. Cannot be used with --readonly or 1191N/A --nasty Instruct the server to misbehave. At random intervals 1191N/A it will time-out, send bad responses, hang up on 1191N/A clients, and generally be hostile. The option 1191N/A takes a value (1 to 100) for how nasty the server 589N/A --proxy-base The url to use as the base for generating internal 589N/A --readonly Read-only operation; modifying operations disallowed. 589N/A Cannot be used with --mirror or --rebuild. 589N/A --rebuild Re-build the catalog from pkgs in depot. Cannot be 589N/A used with --mirror or --readonly. 1431N/A --set-property Used to specify initial repository configuration 1431N/A property values or to update existing ones; can 1431N/A be specified multiple times. If used with --readonly 1431N/A this acts as a temporary override. 1937N/A --socket-path The absolute pathname to a Unix domain socket. 1937N/A If this option is specified, the depot will answer 1937N/A connections through the UNIX socket instead of over IP. 812N/A --ssl-cert-file The absolute pathname to a PEM-encoded Certificate file. 812N/A This option must be used with --ssl-key-file. Usage of 812N/A this option will cause the depot to only respond to SSL 812N/A requests on the provided port. 812N/A --ssl-dialog Specifies what method should be used to obtain the 812N/A passphrase needed to decrypt the file specified by 812N/A --ssl-key-file. Supported values are: builtin, 812N/A --ssl-key-file The absolute pathname to a PEM-encoded Private Key file. 812N/A This option must be used with --ssl-cert-file. Usage of 812N/A this option will cause the depot to only respond to SSL 812N/A requests on the provided port. 1475N/A The maximum size of the indexer sort file. Used to 1475N/A limit the amount of RAM the depot uses for indexing, 975N/A --writable-root The path to a directory to which the program has write 975N/A access. Used with --readonly to allow server to 975N/A create needed files, such as search indices, without 975N/A needing write access to the package information. 1633N/A PKG_REPO Used as default repo_dir if -d not provided. 1633N/A PKG_DEPOT_CONTENT Used as default content_root if --content-root 429N/A """Option exception. """ 466N/A # By default, if the destination for a particular log type is not 466N/A # specified, this is where we will send the output. 466N/A # If stdout is a tty, then send access output there by default instead 1902N/A "debug=",
"disable-ops=",
"exit-ready",
"file-root=",
1902N/A "help",
"llmirror",
"mirror",
"nasty=",
"proxy-base=",
1902N/A "readonly",
"rebuild",
"refresh-index",
"set-property=",
1937N/A "socket-path=",
"ssl-cert-file=",
"ssl-dialog=",
1937N/A "ssl-key-file=",
"sort-file-max-size=",
"writable-root="]
858N/A "A debug feature must be specified." 858N/A # A list of features can be specified using a 858N/A # "," or any whitespace character as separators. 858N/A "Invalid debug feature: " \
1431N/A "An argument must be specified." 466N/A "You must specify a log " \
1191N/A "for nasty option.\n Please " \
1191N/A "choose a value between 1 and 100." 1431N/A "the form '<section.property>=" 589N/A # Attempt to decompose the url provided into 589N/A # its base parts. This is done so we can 589N/A # remove any scheme information since we 765N/A "determine the hostname from " \
765N/A "the provided URL; please use a " \
765N/A "and https are the only supported " \
765N/A # Rebuild the url with the sanitized components. 429N/A # Note: This argument is for internal use 429N/A # only. It's used when pkg.depotd is reexecing 429N/A # itself and needs to know that's the case. 429N/A # This flag is purposefully omitted in usage. 429N/A # The supported way to forcefully reindex is to 429N/A # kill any pkg.depot using that directory, 429N/A # remove the index directory, and restart the 429N/A # pkg.depot process. The index will be rebuilt 429N/A # automatically on startup. 812N/A "the Certificate file must be " \
812N/A "pathname is not a file." 812N/A "the Private Key file must be " \
812N/A "pathname is not a file." 812N/A "specified. Expected: builtin, " \
873N/A # Don't allow a somewhat 873N/A # insecure authentication method 873N/A "not a supported dialog " \
873N/A "type for this operating " \
812N/A "file path specified for " \
1475N/A "a maximum sort file size." 466N/A usage(
"pkg.depotd: illegal option value: %s specified " \
1542N/A usage(
"--add-content cannot be used with --rebuild")
466N/A usage(
"--refresh-index cannot be used with --rebuild")
1633N/A usage(
"--readonly and --mirror cannot be used with --rebuild " 1020N/A usage(
"--mirror cannot be used with --refresh-index")
1020N/A usage(
"--readonly can only be used with --refresh-index if " 1902N/A usage(
"At least one of PKG_REPO, -d, or --file-root" 812N/A usage(
"The --ssl-cert-file and --ssl-key-file options must " 812N/A "must both be provided when using either option.")
812N/A # If they didn't already specify a particular port, use the 812N/A # default SSL port instead. 429N/A # If the program is going to reindex, the port is irrelevant since 429N/A # the program will not bind to a port. 1836N/A emsg(
"pkg.depotd: unable to bind to the specified " 1542N/A # Not applicable if we're not going to serve content 1836N/A "executing [%s]; unable to obtain the " 1836N/A "passphrase needed to decrypt the SSL " 812N/A # The key file requires decryption, but the user has requested 812N/A # exec-based authentication, so it will have to be decoded first 812N/A # to an un-named temporary file. 1836N/A emsg(
"pkg.depotd: unable to read the SSL private key " 1836N/A emsg(
"pkg.depotd: authentication or cryptography " 1836N/A "failure while attempting to decode\nthe SSL " 812N/A # Redirect the server to the decrypted key file. 452N/A # Setup our global configuration. 382N/A "environment":
"production",
858N/A "server.shutdown_timeout":
0,
858N/A # Despite its name, this only logs headers when there is an 858N/A # error; it's redundant with the debug feature enabled. 858N/A # Causes the headers of every request to be logged to the error 858N/A # log; even if an exception occurs. 466N/A "param":
"log.error_file",
466N/A "param":
"log.access_file",
466N/A # Since we've replaced cherrypy's log handler with our 466N/A # own, we don't want the output directed to a file. 612N/A # Now that our logging, etc. has been setup, it's safe to perform any 612N/A # remaining preparation. 1431N/A # Initialize repository state. 1431N/A emsg(
"pkg.depotd: repository configuration error: %s" %
_e)
1431N/A emsg(
"Please use the --set-property option to provide a value, " 1431N/A "or update the cfg_cache file for the repository to " 1431N/A emsg(
"pkg.depotd: repository configuration error: %s" %
_e)
1431N/A # Initializing the repository above updated search indices 1431N/A # as needed; nothing left to do, so exit. 1542N/A # ready to start depot; exit now if requested 451N/A # Now build our site configuration. 452N/A # We have to override cherrypy's default response_class so that 452N/A # we have access to the write() callable to stream data 452N/A # directly to the client. 451N/A # This changes the base URL for our server, and is primarily 451N/A # intended to allow our depot process to operate behind Apache 451N/A # or some other webserver process. 451N/A # Visit the following URL for more information: 451N/A "tools.proxy.local":
"",
451N/A # Now merge or add our proxy configuration information into the 451N/A # existing configuration. 812N/A emsg(
"pkg.depotd: unknown error starting depot server, " \
812N/A "illegal option value specified?")