public class MetricsUtil
extends java.lang.Object
Modifier and Type | Class and Description |
---|---|
static class |
MetricsUtil.ReadableColMetricsStruct
A struct of readable metric values for a primitive column
|
static class |
MetricsUtil.ReadableMetricColDefinition
Fixed definition of a readable metric column, ie a mapping of a raw metric to a readable metric
|
static class |
MetricsUtil.ReadableMetricsStruct
A struct, consisting of all
MetricsUtil.ReadableColMetricsStruct for all primitive columns of the
table |
Modifier and Type | Field and Description |
---|---|
static java.util.List<MetricsUtil.ReadableMetricColDefinition> |
READABLE_METRIC_COLS |
static java.lang.String |
READABLE_METRICS |
Modifier and Type | Method and Description |
---|---|
static java.util.Map<java.lang.Integer,java.lang.Long> |
createNanValueCounts(java.util.stream.Stream<FieldMetrics<?>> fieldMetrics,
MetricsConfig metricsConfig,
Schema inputSchema)
Construct mapping relationship between column id to NaN value counts from input metrics and
metrics config.
|
static MetricsModes.MetricsMode |
metricsMode(Schema inputSchema,
MetricsConfig metricsConfig,
int fieldId)
Extract MetricsMode for the given field id from metrics config.
|
static Schema |
readableMetricsSchema(Schema dataTableSchema,
Schema metadataTableSchema)
Calculates a dynamic schema for readable_metrics to add to metadata tables.
|
static MetricsUtil.ReadableMetricsStruct |
readableMetricsStruct(Schema schema,
ContentFile<?> file,
Types.StructType projectedSchema)
Return a readable metrics struct row from file metadata
|
public static final java.util.List<MetricsUtil.ReadableMetricColDefinition> READABLE_METRIC_COLS
public static final java.lang.String READABLE_METRICS
public static java.util.Map<java.lang.Integer,java.lang.Long> createNanValueCounts(java.util.stream.Stream<FieldMetrics<?>> fieldMetrics, MetricsConfig metricsConfig, Schema inputSchema)
public static MetricsModes.MetricsMode metricsMode(Schema inputSchema, MetricsConfig metricsConfig, int fieldId)
public static Schema readableMetricsSchema(Schema dataTableSchema, Schema metadataTableSchema)
MetricsUtil.ReadableColMetricsStruct
, composed of MetricsUtil.ReadableMetricsStruct
for all
primitive columns in the data tabledataTableSchema
- schema of data tablemetadataTableSchema
- schema of existing metadata table (to ensure id uniqueness)public static MetricsUtil.ReadableMetricsStruct readableMetricsStruct(Schema schema, ContentFile<?> file, Types.StructType projectedSchema)
schema
- schema of original data tablefile
- content file with metricsprojectedSchema
- user requested projectionMetricsUtil.ReadableMetricsStruct