Logo Search packages:      
Sourcecode: hbase version File versions  Download package

org::apache::hadoop::hbase::migration::nineteen::regionserver::HStoreFile Class Reference

Inheritance diagram for org::apache::hadoop::hbase::migration::nineteen::regionserver::HStoreFile:


List of all members.

Detailed Description

A HStore data file. HStores usually have one or more of these files. They are produced by flushing the memcache to disk.

This one has been doctored to be used in migrations. Private and protecteds have been made public, etc.

Each HStore maintains a bunch of different data files. The filename is a mix of the parent dir, the region name, the column name, and a file identifier. The name may also be a reference to a store file located elsewhere. This class handles all that path-building stuff for you.

An HStoreFile usually tracks 4 things: its parent dir, the region identifier, the column family, and the file identifier. If you know those four things, you know how to obtain the right HStoreFile. HStoreFiles may also reference store files in another region serving either from the top-half of the remote file or from the bottom-half. Such references are made fast splitting regions.

Plain HStoreFiles are named for a randomly generated id as in: 1278437856009925445 A file by this name is made in both the mapfiles and info subdirectories of a HStore columnfamily directoy: E.g. If the column family is 'anchor:', then under the region directory there is a subdirectory named 'anchor' within which is a 'mapfiles' and 'info' subdirectory. In each will be found a file named something like 1278437856009925445, one to hold the data in 'mapfiles' and one under 'info' that holds the sequence id for this store file.

References to store files located over in some other region look like this: 1278437856009925445.hbaserepository,qAReLZD-OyQORZWq_vqR1k==,959247014679548184: i.e. an id followed by the name of the referenced region. The data ('mapfiles') of HStoreFile references are empty. The accompanying info file contains the midkey, the id of the remote store we're referencing and whether we're to serve the top or bottom region of the remote store file. Note, a region is not splitable if it has instances of store file references (References are cleaned up by compactions).

When merging or splitting HRegions, we might want to modify one of the params for an HStoreFile (effectively moving it elsewhere).

Definition at line 88 of file HStoreFile.java.

Public Types

enum  Modify {

Public Member Functions

void delete () throws IOException
getReader (final FileSystem fs, final boolean bloomFilter, final boolean blockCacheEnabled) throws IOException
MapFile.Writer getWriter (final FileSystem fs, final SequenceFile.CompressionType compression, final boolean bloomFilter, int nrows) throws IOException
 HStoreFile (HBaseConfiguration conf, FileSystem fs, Path basedir, final int encodedName, byte[] colFamily, long fileId, final Reference ref) throws IOException
synchronized long indexLength () throws IOException
boolean isMajorCompaction ()
long length () throws IOException
long loadInfo (final FileSystem filesystem) throws IOException
boolean rename (final FileSystem fs, final HStoreFile hsf) throws IOException
String toString ()

Static Public Member Functions

static Path getFilterDir (Path dir, int encodedRegionName, final byte[] f)
static Path getInfoDir (Path dir, int encodedRegionName, byte[] f)
static Path getMapDir (Path dir, int encodedRegionName, final byte[] f)
static boolean isReference (final Path p)

Static Public Attributes

static final int ALL_VERSIONS = Integer.MAX_VALUE
static final byte[] CATALOG_FAMILY = Bytes.toBytes(CATALOG_FAMILY_STR)
static final String CATALOG_FAMILY_STR = "info"
static final byte[] CATALOG_HISTORIAN_FAMILY = Bytes.toBytes("historian")
static final String CLUSTER_DISTRIBUTED = "hbase.cluster.distributed"
static final String CLUSTER_IS_DISTRIBUTED = "true"
static final String CLUSTER_IS_LOCAL = "false"
static final String DEFAULT_HOST = ""
static final int DEFAULT_MASTER_INFOPORT = 60010
static final int DEFAULT_MASTER_PORT = 60000
static final long DEFAULT_MAX_FILE_SIZE = 256 * 1024 * 1024
static final String DEFAULT_REGION_SERVER_CLASS = HRegionInterface.class.getName()
static final int DEFAULT_REGIONSERVER_INFOPORT = 60030
static final int DEFAULT_REGIONSERVER_PORT = 60020
static final int DEFAULT_SIZE_RESERVATION_BLOCK = 1024 * 1024 * 5
static final int DEFAULT_ZOOKEEPER_PAUSE = 2 * 1000
static final int DEFAULT_ZOOKEEPER_RETRIES = 5
static final byte[] EMPTY_BYTE_ARRAY = new byte [0]
static final byte[] EMPTY_END_ROW = EMPTY_START_ROW
static final byte[] EMPTY_START_ROW = EMPTY_BYTE_ARRAY
static final String FILE_SYSTEM_VERSION = "7"
static final int FOREVER = Integer.MAX_VALUE
static String HBASE_CLIENT_SCANNER_MAX_RESULT_SIZE_KEY = "hbase.client.scanner.max.result.size"
static final String HBASE_DIR = "hbase.rootdir"
static final String HREGION_COMPACTIONDIR_NAME = "compaction.dir"
static final String HREGION_LOGDIR_NAME = ".logs"
static final String HREGION_OLDLOGFILE_NAME = "oldlogfile.log"
static final String IN_MEMORY = "IN_MEMORY"
static final byte[] LAST_ROW = EMPTY_BYTE_ARRAY
static final long LATEST_TIMESTAMP = Long.MAX_VALUE
static final byte[] LATEST_TIMESTAMP_BYTES = Bytes.toBytes(LATEST_TIMESTAMP)
static final String MAJOR_COMPACTION_PERIOD = "hbase.hregion.majorcompaction"
static final String MASTER_PORT = "hbase.master.port"
static final int MAX_ROW_LENGTH = Short.MAX_VALUE
static final int MAXIMUM_VALUE_LENGTH = Integer.MAX_VALUE
static final int META_ROW_DELIMITER = ','
static final byte[] META_TABLE_NAME = Bytes.toBytes(".META.")
static final String NAME = "NAME"
static final String NINES = "99999999999999"
static final String REGION_IMPL = "hbase.hregion.impl"
static final String REGION_SERVER_CLASS = "hbase.regionserver.class"
static final String REGION_SERVER_IMPL = "hbase.regionserver.impl"
static final byte[] REGIONINFO_QUALIFIER = Bytes.toBytes("regioninfo")
static final String REGIONSERVER_PORT = "hbase.regionserver.port"
static int RETRY_BACKOFF [] = { 1, 1, 1, 2, 2, 4, 4, 8, 16, 32 }
static final byte[] ROOT_TABLE_NAME = Bytes.toBytes("-ROOT-")
static final byte[] SERVER_QUALIFIER = Bytes.toBytes("server")
static final byte[] SPLITA_QUALIFIER = Bytes.toBytes("splitA")
static final byte[] SPLITB_QUALIFIER = Bytes.toBytes("splitB")
static final byte[] STARTCODE_QUALIFIER = Bytes.toBytes("serverstartcode")
static final String THREAD_WAKE_FREQUENCY = "hbase.server.thread.wakefrequency"
static final String UTF8_ENCODING = "UTF-8"
static final String VERSION_FILE_NAME = "hbase.version"
static final String VERSIONS = "VERSIONS"
static final int WEEK_IN_SECONDS = 7 * 24 * 3600
static final String ZEROES = "00000000000000"
static final String ZOOKEEPER_CONFIG_NAME = "zoo.cfg"
static final String ZOOKEEPER_PAUSE = "zookeeper.pause"
static final String ZOOKEEPER_QUORUM = "hbase.zookeeper.quorum"
static final String ZOOKEEPER_RETRIES = "zookeeper.retries"

Package Functions

byte[] getColFamily ()
int getEncodedRegionName ()
long getFileId ()
Path getInfoFilePath ()
Path getMapFilePath ()
Reference getReference ()
 HStoreFile (HBaseConfiguration conf, FileSystem fs, Path basedir, final int encodedName, byte[] colFamily, long fileId, final Reference ref, final boolean mc) throws IOException
boolean isReference ()
void splitStoreFile (final HStoreFile dstA, final HStoreFile dstB, final FileSystem fs) throws IOException
void writeInfo (final FileSystem filesystem, final long infonum, final boolean mc) throws IOException
void writeInfo (final FileSystem filesystem, final long infonum) throws IOException
void writeReferenceFiles (final FileSystem fs) throws IOException

Static Package Functions

static Reference readSplitInfo (final Path p, final FileSystem fs) throws IOException

Static Package Attributes

static final String HSTORE_DATFILE_DIR = "mapfiles"
static final String HSTORE_FILTER_DIR = "filter"
static final String HSTORE_INFO_DIR = "info"
static final byte INFO_SEQ_NUM = 0
static final Log LOG = LogFactory.getLog(HStoreFile.class.getName())
static final byte MAJOR_COMPACTION = INFO_SEQ_NUM + 1
static final Long ZERO_L = Long.valueOf(0L)

Private Member Functions

void createOrFail (final FileSystem fs, final Path p) throws IOException
Path getInfoFilePath (final int encodedName, final long fid, final int ern)
Path getInfoFilePath (final int encodedName, final long fid)
Path getMapFilePath (final int encodedName, final long fid, final int ern)
Path getMapFilePath (final int encodedName, final long fid)
Path getMapFilePath (final Reference r)
void writeSplitInfo (final FileSystem fs) throws IOException

Static Private Member Functions

static String createHStoreFilename (final long fid, final int encodedRegionName)
static Path getFamilySubDir (final Path base, final int encodedRegionName, final byte[] f, final String subdir)
static boolean isReference (final Path p, final Matcher m)

Private Attributes

final Path basedir
final byte[] colFamily
final HBaseConfiguration conf
final int encodedRegionName
final long fileId
final FileSystem fs
final HRegionInfo hri
long indexLength
boolean majorCompaction = false
final Reference reference

Static Private Attributes

static final Random rand = new Random()
static final Pattern REF_NAME_PARSER

The documentation for this class was generated from the following file:

Generated by  Doxygen 1.6.0   Back to index