刁赢龙;盛万兴;刘科研;何开元;孟晓丽;

• 1:中国电力科学研究院

摘要(Abstract)：

为减少数据缓存成本,提高负荷数据在配电网规划设计、智能分析等领域的可用性,充分有效地对大规模、混杂、不精确的监测或采集负荷数据进行在线清洗,保证每个周期的时序数据得到一致的偏差检测和精确修复,在分析不同类型异常负荷数据产生原因和分布特点的基础上,提出一种面向大规模配电网负荷数据的在线清洗与修复方法,包括基于密度的负荷数据流异常辨识方法和基于协同过滤推荐算法的负荷数据修复方法。为突破配电网负荷大数据在线分析性能瓶颈,还在Hadoop平台上给出相应的分布式并行解决方案,通过使用实际配电网运行中的负荷数据进行验证,结果表明所提算法和框架能够有效预处理配电网负荷数据,具有实际应用价值。

关键词(KeyWords)： 数据清洗;流数据;大规模配电网;在线清洗

Abstract:

Keywords:

基金项目(Foundation): 国家电网公司科技项目(EPRIPDKJ[2014]3763号)~~

作者(Author): 刁赢龙;盛万兴;刘科研;何开元;孟晓丽;

Email:

DOI: 10.13335/j.1000-3673.pst.2015.11.018

参考文献(References)：

• [1]赵腾,张焰,张东霞.智能配电网大数据应用技术与前景分析[J].电网技术,2014,38(12):3305-3312.Zhao Teng,Zhang Yan,Zhang Dongxia.Application technology of big data in smart distribution grid and its prospect analysis[J].Power System Technology,2014,38(12):3305-3312(in Chinese).
• [2]刘科研,盛万兴,张东霞,等.智能配电网大数据应用需求和场景分析研究[J].中国电机工程学报,2010,35(2):287-293.Liu Keyan,Sheng Wanxing,Zhang Dongxia,et al.Big data application requirements and scenario analysis in smart distribution network[J].Proceedings of the CSEE,2010,35(2):287-293(in Chinese).
• [3]张东霞,苗新,刘丽平,等.智能电网大数据技术发展研究[J].中国电机工程学报,2015,35(1):2-12.Zhang Dongxia,Miao Xin,Liu Liping,et al.Research on development strategy for smart grid big data[J].Proceedings of the CSEE,2015,35(1):2-12(in Chinese).
• [4]宋亚奇,周国亮,朱永利.智能电网大数据处理技术现状与挑战[J].电网技术.2013,37(4):927-935.Song Yaqi,Zhou Guoliang,Zhu Yongli.Present status and challenges of big data processing in smart grid[J].Power System Technology,2013,37(4):927-935(in Chinese).
• [5]毛李帆,姚建刚,金永顺.中长期负荷预测的异常数据辨识与缺失数据处理[J].电网技术,2010,34(7):148-153.Mao Lifan,Yao Jiangang,Jin Yongshun,et al.Abnormal data identification and missing data filling in medium-and long-term load forecasting[J].Power System Technology,2010,34(7):148-153(in Chinese).
• [6]张乐.年度负荷预测工作中的数据还原与修正[J].电网技术,2007,31(S1):233-234.Zhang Le.Data recovery and correction in annual load-forecasting[J].Power System Technology,2007,31(S1):233-234(in Chinese).
• [7]王雁平,乐春峡.电力系统负荷建模的数据预处理技术[J].电网技术,2007,31(S2):292-294.Wang Yanping,Yue Chunxia.A data pretreatment technique about power system load modeling[J].Power System Technology,2007,31(S2):292-294(in Chinese).
• [8]毛李帆,江岳春,龙瑞华,等.基于偏最小二乘回归分析的中长期电力负荷预测[J].电网技术,2008,32(19):71-77.Mao Lifan,Jiang Yuechun,Long Ruihua,et al.Medium and long term load forecasting based on partial least squares regression analysis[J].Power System Technology,2008,32(19):71-77(in Chinese).
• [9]张沛,吴潇雨,和敬涵.大数据技术在主动配电网中的应用综述[J].电力建设,2015,36(1):52-59.Zhang Pei,Wu Xiaoyu,He Jinghan.Review on big data technology applied in active distribution network[J].Power Construction,2015,36(1):52-59(in Chinese).
• [10]严英杰,盛戈皞,陈玉峰,等.基于大数据分析的输变电设备状态数据异常检测方法[J].中国电机工程学报,2015,35(1):52-59.Yan Yingjie,Sheng Gehao,Chen Yufeng,et al.An method for anomaly detection of state information of power equipment based on big data analysis[J].Proceedings of the CSEE,2015,35(1):52-59(in Chinese).
• [11]王宁玲,付鹏,陈德刚,等.大数据分析方法在厂级负荷分配中的应用[J].中国电机工程学报,2015,35(1):68-73.Wang Ningling,Fu Peng,Chen Degang,et al.Application of big data analytics in plant-level load dispatching of power plant[J].Proceedings of the CSEE,2015,35(1):68-73(in Chinese).
• [12]张智晟,孙雅明,张世英,等.基于数据挖掘多层次细节分解的负荷序列聚类分析[J].电网技术,2006,30(2):51-56.Zhang Zhisheng,Sun Yaming,Zhang Shiying,et al.Clustering analysis of electric load series using clustering algorithm of multihierarchy and detailed decomposition based on data mining[J].Power System Technology,2006,30(2):51-56(in Chinese).
• [13]高静,段会川.JSON数据传输效率研究[J].计算机工程与设计,2011,32(7):2267-2270.Gao Jing,Duan Huichuan.Research on data transmission efficiency of JSON[J].Computer Engineering and Design,2011,32(7):2267-2270(in Chinese).
• [14]严英杰,盛戈皞,陈玉峰,等.基于时间序列分析的输变电设备状态大数据清洗方法[J].电力系统自动化,2015,39(7):138-144.Yan Yingjie,Sheng Gehao,Chen Yufeng,et al.Cleaning method for big data of power transmission and transformation equipment state based on times sequence analysis[J].Microcomputer Information,2015,39(7):138-144(in Chinese).
• [15]顾民,葛良全,熊文贤,等.基于神经网络的电力负荷不良数据的清洗[J].微计算机信息,2005,23(7-3):60-64.Gu Min,Ge Liangquan,Xiong Wenxian,et al.Cleaning for bad electric load data based on neural network[J].Microcomputer Information,2005,23(7-3):60-64(in Chinese).
• [16]张晓星,程其云,周湶,等.基于数据挖掘的电力负荷脏数据动态智能清洗[J].电力系统自动化,2005,29(8):60-64.Zhang Xiaoxing,Cheng Qiyun,Zhou Quan,et al.Dynamic intelligent cleaning for dirty electric load data based on data mining[J].Automation of Electric Power Systems,2005,29(8):60-64(in Chinese).