imgrpkg

Overview ¶

Package imgrpkg implements the server side Inode Management for ProxyFS volumes. While the package provides a small set of Go-callable APIs, the bulk of its functionality is accessed via package retryrpc-exposed RPCs. While these RPCs reference active volumes known to an imgrpkg instance, a RESTful API is provided to specify those active volumes.

Note that func's listed under type RetryRPCServerStruct are the RPCs issued by the client side via package retryrpc connections (and, thus, not intended to be called directly).

To configure an imgrpkg instance, Start() is called passing, as the sole argument, a package conf ConfMap. Here is a sample .conf file:

[IMGR]
PublicIPAddr:                         imgr
PrivateIPAddr:                        imgr
RetryRPCPort:                         32356
HTTPServerPort:                       15346

CheckPointIPAddrs:                           # List of ickpt cluster instance IPAddr's
CheckPointPort:                       33123  # Only required if CheckPointIPAddrs is non-empty
CheckPointCACertFilePath:                    # Defaults to /dev/null disabling TLS
CheckPointRetryDelay:                 100ms  # Only required if CheckPointIPAddrs is non-empty
CheckPointRetryExpBackoff:            2      # Only required if CheckPointIPAddrs is non-empty
CheckPointRetryLimit:                 4      # Only required if CheckPointIPAddrs is non-empty
CheckPointTimeout:                    10m    # Only required if CheckPointIPAddrs is non-empty
CheckPointConnectionPoolSize:         10     # Only required if CheckPointIPAddrs is non-empty

RetryRPCTTLCompleted:                 10m
RetryRPCAckTrim:                      100ms
RetryRPCDeadlineIO:                   60s
RetryRPCKeepAlivePeriod:              60s

RetryRPCCertFilePath:                        # If both RetryRPC{Cert|Key}FilePath are missing or empty,
RetryRPCKeyFilePath:                         #   non-TLS RetryRPC will be selected; otherwise TLS will be used

CheckPointInterval:                   10s

AuthTokenCheckInterval:               1m

FetchNonceRangeToReturn:              100

MountLimit:                           10000
OpenFileLimit:                        100000

MinLeaseDuration:                     250ms
LeaseInterruptInterval:               250ms
LeaseInterruptLimit:                  20
LeaseEvictLowLimit:                   100000
LeaseEvictHighLimit:                  100010

SwiftRetryDelay:                      100ms
SwiftRetryExpBackoff:                 2
SwiftRetryLimit:                      4

SwiftTimeout:                         10m
SwiftConnectionPoolSize:              128

ParallelObjectDeletePerVolumeLimit:   100

InodeTableCacheEvictLowLimit:         10000
InodeTableCacheEvictHighLimit:        10010

InodeTableMaxInodesPerBPlusTreePage:  2048
RootDirMaxDirEntriesPerBPlusTreePage: 1024

LogFilePath:                                 # imgr.log
LogToConsole:                         true   # false
TraceEnabled:                         false
RetryRPCLogEnabled:                   false

Most of the config keys are required and must have values. One exception is LogFilePath that will default to "" and, hence, cause logging to not go to a file. This might typically be used when LogToConsole is set to true.

The RetryRPC{Cert|Key}FilePath keys are optional and, if provided may be empty. In such cases, the retryrpc package will be configured to use TCP. If, however, they are present and provide a path or paths to valid Cert|Key files, the retryrpc package will be configured to use TLS. In any event, the RPCs will be available via <PublicIPAddr>:<RetryRPCPort>.

The RESTful API is provided by an embedded HTTP Server (at URL http://<PrivateIPAddr>:<HTTPServerPort>) responds to the following:

DELETE /keepalive

This will disable the keep alive mechanism.

DELETE /volume/<volumeName>

This will cause the specified <volumeName> to no longer be served. Note that this does not actually affect the contents of the associated Container.

GET /config

This will return a JSON document that matches the conf.ConfMap used to launch this package.

GET /keepalive

This will return the configured keepalive duration if any. If the keepalive mechanism has not been enabled, a 404 Not Found will be returned. If the keepalive mechanism has been enabled, this is the highest performing way to reset the countdown timer.

GET /stats

This will return a raw bucketstats dump.

GET /version

This will return the imgr version.

GET /volume

This will return a JSON document containing an array of volumes currently being served with details about each.

GET /volume/<volumeName>

This will return a JSON document containing only the specified <volumeName> details (assuming it is currently being served).

POST /volume
Content-Type: application/json

{
   "StorageURL": "http://172.28.128.2:8080/v1/AUTH_test/con",
   "AuthToken" : "AUTH_tk0123456789abcde0123456789abcdef0"
}

This will cause the specified StorageURL to be formatted. The StorageURL specified in the JSON document content identifies the Container for format. The AuthToken in the JSON document content provides the authentication to use during the formatting process.

PUT /keepalive/<duration>

This will configure the keep alive mechanism to start a count down timer for the specified duration after which all served volumes will no longer be served. The expiration of this count down timer can be avoided by GET-ing (/keepalive) the current duration, PUT-ing (/keepalive/<duration) a fresh keep alive duration, GET-ing (/volume) the list of volumes currently served, PUT-ing (/volume/<volumeName>) a new volume, or by DELETE-ing (/keepalive) the keep alive mechanism. Note that if tje keep alive mechanism ever expired, the current duratiom will be reset to zero (i.e. no expiration).

PUT /volume/<volumeName>
Content-Type: application/json

{
   "StorageURL": "http://172.28.128.2:8080/v1/AUTH_test/con"
}

This will cause the specified <volumeName> to be served. The StorageURL specified in the JSON document content identifies the Container to serve. Clients will each supply an AuthToken in their Mount/RenewMount requests that will be used to access the Container.

PUT /volume/<volumeName>
Content-Type: application/json

{
   "StorageURL": "http://172.28.128.2:8080/v1/AUTH_test/con",
   "AuthToken" : "AUTH_tk0123456789abcde0123456789abcdef0"
}

This will cause the specified <volumeName> to be served. The StorageURL specified in the JSON document content identifies the Container to serve. Clients will each supply an AuthToken in their Mount/RenewMount requests that will be used to access the Container. As a debugging aid, and in the case where no Clients have <volumeName> mounted, the AuthToken in the JSON document content will be used to access the Container.