Chung-Ang
University Scientists Develop New Framework for Home Energy Management Systems
This distributed, privacy-aware machine
learning approach will help smart homes optimize the scheduling of appliances
and reduce electricity bills
Home
energy management systems (HEMSs) are used in smart homes to calculate
optimized schedules for electric home appliances, make energy use more
efficient, and reduce electricity bills. In a recent study, Korean scientists
developed a framework for training HEMSs using a data-driven machine learning
approach called federated reinforced learning. Their strategy allows HEMSs to
be trained in a distributed manner without computationally intensive steps nor
privacy concerns about the users’ energy consumption data.
Training home energy management systems in a federated (distributed) manner can help keep users’ data private while lowering the computational complexity of the machine learning algorithms involved.
Image courtesy:
Shutterstock
As we advance steadily into the era of smart grids and smart homes, new
opportunities to use energy more efficiently appear. For example, many cities
are implementing demand response programs, in which the cost of electricity is
lowered outside of peak hours to incentivize people to use energy when there is
less load on the grid. Another example is the coordination of distributed
generation resources such as solar panels and their battery systems, which has
to be optimized according to weather conditions and the supply and demand of
energy in real time.
Fortunately, most of the decisions and actions needed to optimize our
energy use can be delegated to home energy management systems (HEMSs). These
devices are designed to efficiently manage the energy consumption of home
appliances by doing things such as scheduling the time when the washing machine
should start and strategically turning the air conditioner (AC) on and off. In
general, the main objective of a HEMS is to minimize electricity bills while
also taking the user’s preferences and comfort into account.
One way to program these devices is to use
hand-crafted models, which rely on abstract equations that represent the
appliances and distributed energy resources. Such models and optimization
methods, however, are not very versatile and tend to give suboptimal solutions.
Another approach is to use centralized machine learning, where data from
thousands of users is collected, sent to a central server, and used to train a
model from the ground up. This strategy poses two problems. First, sending and
processing large volumes of user data is a challenge in and of itself; it’s a
costly and computationally intensive endeavor. Second, the central server could
become a juicy target for hackers wanting to steal private information about people’s
energy consumption patterns.
Against this backdrop, Associate Professor Dae-Hyun
Choi and PhD student Sangyoon Lee from Chung-Ang University, South Korea, have
come up with a novel data-driven strategy to tackle both of these problems
simultaneously. As explained in a recent paper, the researchers developed a
framework for HEMS based on federated deep reinforcement learning (F-DRL),
combining the advantages of various machine learning techniques. The study was
made available online on November 3, 2020, and published in Volume 18, Issue 1
of IEEE
Transactions on Industrial Informatics in January 2022.
The key word to note in F-DRL is ‘federated,’ which
indicates a decentralized form of machine learning. In the proposed framework,
each home has a HEMS connected to various appliances and devices, namely a
washing machine, a solar panel, an AC, and an energy storage system. Each HEMS
collects data about its users’ energy consumption and tries to optimize a
schedule for the appliances by creating a local model. These local models are
all uploaded to a global server, which averages them to produce a global model.
Afterwards, each HEMS replaces its local model with the global model and proceeds
to train it once again using local data. This process is repeated several times,
progressively improving the accuracy of both global and local models. “In a
conventional centralized DRL model, the global server must have access to the data
of all local devices to generate the model of the global system. This results
in data privacy concerns for local devices,” explains Associate Professor
Choi, “However, in our federated DRL method, the system does not require the
sharing of user data because only the parameters of local and global model are exchanged.
In turn, this helps prevent local data leakages and protect the users’ privacy.”
The researchers tested their approach through simulations,
showcasing its optimal performance when scheduling the operation of various
appliances at different homes. “To the best of our knowledge, this is the
first HEMS framework based on federated DRL that can manage the energy
consumption of multiple smart homes and ensure the comfort of the consumers
while taking their preferences into account in a distributed manner,”
remarks Associate Professor Choi, satisfied with the results. In addition to its
low computational complexity and relatively fast training process, the proposed
framework can easily support the addition of more appliances at each house.
Associate Professor Choi envisions a more
comprehensive version of this framework that also takes into account electric
cars and energy trading between households. Let us hope he meets this goal so
that, in the future, we can let HEMSs optimize our energy consumption without
ever having to worry about our privacy.
Reference
Authors
Title of original paper
Journal |
Sangyoon Lee and
Dae-Hyun Choi
Federated
Reinforcement Learning for Energy Management of Multiple Smart Homes With
Distributed Energy Resources IEEE
Transactions on Industrial Informatics |
|
|
DOI
Affiliation |
10.1109/TII.2020.3035451
School of Electrical
and Electronics Engineering, Chung-ang University |
About Chung-Ang University
Chung-Ang University is a private
comprehensive research university located in Seoul, South Korea. It was started
as a kindergarten in 1918 and attained university status in 1953. It is fully
accredited by the Ministry of Education of Korea. Chung-Ang University conducts
research activities under the slogan of “Justice and Truth.” Its new vision for
completing 100 years is “The Global Creative Leader.” Chung-Ang University
offers undergraduate, postgraduate, and doctoral programs, which encompass a
law school, management program, and medical school; it has 16 undergraduate and
graduate schools each. Chung-Ang University’s culture and arts programs are
considered the best in Korea.
Website: https://neweng.cau.ac.kr/index.do
About Associate Professor Dae-Hyun Choi
Dae-Hyun
Choi is an Associate Professor at the School of Electrical and Electronics
Engineering, Chung-Ang University, Seoul, South Korea. His group researches
smart grid application using deep reinforcement learning, cyber-physical
security aspects of smart grids, and the theory and applications of
cyber-physical energy systems. Before joining Chung-Ang University, he was a
Researcher at Korea Telecom, South Korea, where he worked on designing and
implementing home network systems. He also worked at LG Electronics, South
Korea, where he developed home energy management systems. In 2014, Dae-Hyun
Choi received a PhD in electrical and computer engineering from Texas A&M
University, College Station, TX, USA.