ics.AUC
tensorflow2.*常⽤
ics.AUC(
num_thresholds=200, curve='ROC',
summation_method='interpolation', name=None, dtype=None,
thresholds=None, multi_label=False, num_labels=None, label_weights=None,
from_logits=False
)
此度量标准创建⽤于计算AUC的四个局部变量 true_positives ， true_negatives ， false_positives 和 false_negatives 。为了离散化AUC曲线，使⽤⼀组线性间隔的阈值来计算成对的召回率和精度值。因此，ROC曲线下⽅的⾯积是通过假阳性率使⽤召回值的⾼度来计算的，⽽PR曲线下⽅的⾯积是通过召回 率使⽤精度值的⾼度来计算的。
该值最终以 auc 形式返回，这是⼀个幂等运算，⽤于计算精度与召回值（使⽤上述变量计算）的离散曲线下的⾯积。
num_thresholds 变量控制离散化具有较⼤的阈值更紧密地逼近真实AUC的数量程度。近似的质量可能会因 num_thresholds ⽽有很⼤差异。
thresholds 参数可⽤于⼿动指定阈值，这些阈值可以更均匀地划分预测。 sample_weight 为 None ，则权重默认为1。使⽤ sample_weight 为0掩盖值。
为了获得最佳结果， predictions 应该在[0，1]范围内⼤致均匀分布，并且不要在0或1附近达到峰值。如果不是这种情况，则AUC近似值的质量可能很差。将 summation_method 设置为“ minoring”或“ majoring”可以通过提供AUC的下限或上限估算值来帮助量化近似值中的误差。
[]
tensorflow1.*常⽤
labels, predictions, weights=None, num_thresholds=200, metrics_collections=None,
updates_collections=None, curve='ROC', name=None,
summation_method='trapezoidal', thresholds=None
)
[]
使⽤tf.estimator时，如果调⽤ Estimator 的 evaluate ⽅法，则 model_fn 会收到 mode = ModeKeys.EVAL。在这种情况下，模型函数必须返回⼀个包含模型损失和⼀个或多个指标（可选）的 tf.estimator.EstimatorSpec。虽然返回指标是可选的，但⼤多数⾃定义Estimator ⾄少会返回⼀个指标。TensorFlow 提供⼀个指标模块 tf.metrics 来计算常⽤指标。 ⼏个常⽤的指标
这些可能只针对⼆分类
⽂档表⽰标签和预测都将转换为bool，因此它只涉及⼆进制分类。也许有可能对这些例⼦进⾏热门编码，它会起作⽤吗？但不确定这⼀点。[]
: Calculates how often predictions matches labels.
The accuracy function creates two local variables, total and count that are used to compute the frequency with
旧金山和约which predictions matches labels. This frequency is ultimately returned as accuracy: an idempotent operation that simply divides total by count.
: Computes the approximate AUC via a Riemann sum.
: Computes average precision@k of predictions with respect to sparse labels.
: Computes the precision of the predictions with respect to the labels. 准确率。 函数会将我们的预测值与真实值进⾏⽐较，即与输⼊函数提供的标签进⾏⽐较。 函数要求标签和预测具有相同的形状。
: Computes precision@k of the predictions with respect to sparse labels.
: Computes the recall of the predictions with respect to the labels.
: Computes recall@k of the predictions with respect to sparse labels.
[]
[]
初始化
这些函数创建的都是local variables，直接初始化时需要使⽤sess.run(tf.local_variables_initializer())⽽不是
tf.global_variables_initializer()。不初始化可能出错：Attempting to use uninitialized value total_confusion_matrix。
参数
1 如果输出的是序列label（如ner模型），则⼀般需要使⽤mask。[]
2 对于分类模型，
2.1 计算precission、recall时，pred_ids需要是one-hot形式，如
labels = [[0, 1, 0],
[1, 0, 0],
[0, 0, 1]]，
[]
[]
note:
1 当然对⽐的pred_ids不能是有负值的logits，否则出错[`predictions` contains negative values] # [Condition x >= 0 did not hold element-wise:] [x (Reshape_2:0) = ] [0 -]。
2 ⾮要改成⾮one-hot形式，如果argmax维度搞错没写或0，输⼊(batch_size, num_labels)，输出本应是(batch_size,)，变成了输出(num_labels,)，⼀般如果num_labels>batch_size不会报错，<；则报错“(batch_size, num_labels) tf_metircs [`labels` out of bound] [Condition x < y did not hold element-wise:]”，但是两者都是错误的。
2.2 计算acc、auc（这个不清楚原理）时则不需要这种转换，直接输⼊即可。
多类分类的测试
计算precission、recall时，pred_ids需要是one-hot形式，如
labels = [[0, 1, 0],
[1, 0, 0],
[0, 0, 1]]，
经⼤规模测试，发现其计算实际上是micro平均，即precission=recall=acc；同时⾃带的这种等价于使⽤下⾯提到的多分类指标评价
predictions=tf.(pred_ids,1))。
返回值
以accuracy的返回值为例：
accuracy: A Tensor representing the accuracy, the value of total divided by count. 准确性调⽤不会使⽤新输⼊更新度量标准，它只使⽤两个局部变量返回值。（具体意思看⽰例1就ok了）
update_op: An operation that increments the total and count variables appropriately and whose value matches accuracy.
Multi-class metrics for Tensorflow: tf_metrics
precision(labels, predictions, num_classes, pos_indices=None, weights=None, average='micro'):
参数：
labels : Tensor of tf.int32 or tf.int64
The true labels 输⼊为shape=(batch,)的⾮one-hot的labels列表。
predictions : Tensor of tf.int32 or tf.int64
The predictions, same shape as labels
num_classes : int
The number of classes
pos_indices : list of int, optional
爱农网
The indices of the positive classes, default is all
weights : Tensor of tf.int32, optional
Mask, must be of compatible shape with labels
average : str, optional
'micro': counts the total number of true positives, false
positives, and false negatives for the classes in
`pos_indices` and infer the metric from it.
'macro': will compute the metric separately for each class in
`pos_indices` and average. Will not account for class
imbalance.
'weighted': will compute the metric separately for each class in
`pos_indices` and perform a weighted average by the total
number of true labels for each class.
recall(labels, predictions, num_classes, pos_indices=None, weights=None, average='micro')
f1(labels, predictions, num_classes, pos_indices=None, weights=None, average='micro')
输⼊如果是one-hot形式，需要转换成预测标签类别
acc, acc_op = tf_metrics.accuracy(labels=tf.argmax(labels, 1), predictions=tf.(logits,1))
⽰例
⽰例1
label_ids = tf.constant([[3, 1, 5]])
pred_ids = tf.constant([[3, 2, 5]])
acc, acc_op = tf.metrics.accuracy(label_ids, pred_ids)
stream_vars = [i for i in tf.local_variables()]
print(stream_vars)
with tf.Session() as sess:宝马z9
sess.run(tf.local_variables_initializer())
print('[total, count]:', sess.run(stream_vars))
print(acc.eval()) # 只使⽤两个局部变量（此时未更新为0）返回值
print(acc_op.eval())
print('[total, count]:', sess.run(stream_vars))
print(acc.eval()) # 只使⽤两个局部变量（此时已更新⾮0）返回值[<tf.Variable 'accuracy/total:0' shape=() dtype=float32_ref>, <tf.Variable 'accuracy/count:0' shape=() dtype=float32_ref>]电脑迷
[total, count]: [0.0, 0.0]
0.0
0.6666667
[total, count]: [2.0, 3.0]
0.6666667
[]
[]
[]
⽰例2
# Compute evaluation metrics.
acc, acc_op = tf.metrics.accuracy(labels=tf.argmax(labels, 1), predictions=tf.argmax(logits,1))⽰例3：多分类
label_ids = tf.constant([[0, 0, 0, 1],
[0, 0, 1, 0],
[1, 0, 0, 0],
[0, 1, 0, 0],
[0, 1, 0, 0]])
pred_ids = tf.constant([[0, 0, 0, 1],
[0, 1, 0, 0],
[1, 0, 0, 0],
[1, 0, 0, 0],
[1, 0, 0, 0]])
num_labels = label_ids.shape[1]
label_arg_ids = tf.argmax(label_ids, 1)
pred_arg_ids = tf.argmax(pred_ids, 1)
# _, tp_op = ue_positives(label_ids, pred_ids)
# _, fp_op = tf.metrics.false_positives(label_ids, pred_ids)
明英苑
_, acc_op = tf.metrics.precision(label_ids, pred_ids)
_, acc_op1 = tf.metrics.accuracy(label_arg_ids, pred_arg_ids)
_, pre_op = tf.metrics.precision(label_ids, pred_ids)
# _, pre_op1 = tf.metrics.precision(label_arg_ids, pred_arg_ids)
_, rec_op = all(label_ids, pred_ids)
# _, rec_op1 = all(label_arg_ids, pred_arg_ids)
# _, pre_op_ = tf_metrics.precision(label_ids, pred_ids, num_labels)
_, pre_op1_ = tf_metrics.precision(label_arg_ids, pred_arg_ids, num_labels, average='macro') # _, rec_op_ = all(label_ids, pred_ids, num_labels)
_, rec_op1_ = all(label_arg_ids, pred_arg_ids, num_labels, average='macro') _, f1_op1_ = tf_metrics.f1(label_arg_ids, pred_arg_ids, num_labels, average='macro')
stream_vars = [i for i in tf.local_variables()]
print(stream_vars)
with tf.Session() as sess:
sess.run(tf.local_variables_initializer())
print(label_arg_ids.eval())
望京南湖中园小学print(pred_arg_ids.eval())
# print(tp_op.eval()) # 2
# print(fp_op.eval()) # 3
print('acc_op:', acc_op.eval())
print('acc_op1:', acc_op1.eval())
print('pre_op:', pre_op.eval())
# print('pre_op1:', pre_op1.eval()) # 1.0
print('rec_op:', rec_op.eval())
# print('rec_op1:', rec_op1.eval()) # 0.5
# print(pre_op_.eval()) # 0.7
print('pre_op1_:', pre_op1_.eval())
# print(rec_op_.eval()) # 0.7
print('rec_op1_:', rec_op1_.eval())
print('f1_op1_:', f1_op1_.eval())
豫公网安备41160202000603
站长QQ:729038198
关于我们
投诉建议