[et_pb_section fb_built=”1″ fullwidth=”on” _builder_version=”4.4.1″][et_pb_fullwidth_post_title meta=”off” featured_placement=”background” _builder_version=”4.7.7″ title_font=”||||||||” title_text_color=”#ff6317″ title_font_size=”3.5em” meta_font=”|300|||||||” meta_text_color=”#ffffff” meta_font_size=”1em” background_enable_color=”off” use_background_color_gradient=”on” background_color_gradient_start=”rgba(248,248,248,0.85)” background_color_gradient_end=”rgba(248,248,248,0.75)” background_color_gradient_overlays_image=”on” min_height=”20vh” height=”400px” custom_padding=”6vh||6vh||false|false” global_module=”403″ locked=”off”][\/et_pb_fullwidth_post_title][\/et_pb_section][et_pb_section fb_built=”1″ admin_label=”intro” _builder_version=”4.7.7″ custom_padding=”|||0px||” locked=”off”][et_pb_row use_custom_gutter=”on” _builder_version=”4.4.6″ custom_margin=”||||false|false” custom_padding=”6vh||||false|false” border_color_left=”rgba(0,0,0,0)”][et_pb_column type=”4_4″ _builder_version=”4.4.1″][et_pb_text _builder_version=”4.7.7″ text_font=”||||||||” text_font_size=”1.1em” text_line_height=”1.6em” quote_font=”|700|||||||” quote_text_align=”left” quote_font_size=”16px” header_2_text_color=”#ff6317″ header_2_font_size=”1.5em” header_2_line_height=”0.9em” header_3_font_size=”23px” header_4_font=”||||||||” header_4_font_size=”16px” header_4_line_height=”1.5em” header_5_font_size=”14px” custom_margin=”||||false|false” custom_padding=”||||false|false” hover_enabled=”0″ border_color_left=”#ff6317″ sticky_enabled=”0″]<\/p>\n
At Berg Software, we created a Dynamic IP solution that would help us overcome the challenges associated with remote work. When working from home (like everyone else these days), Berg Software\u2019s developers use a VPN to get a secure connection to our on-premises infrastructure. In some cases, this is perfectly sufficient. However, when accessing the non-prod infrastructure, things get a bit more challenging:<\/p>\n
We, therefore, imagined a Dynamic IP solution that should do two things:<\/p>\n
Our Dynamic IP solution was an automated, scheduled tool that automates the refreshment of the firewall\/IP list, thus eliminating Ops intervention and reducing deadtimes.<\/p>\n
[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row use_custom_gutter=”on” _builder_version=”4.7.7″ custom_margin=”||||false|false” custom_padding=”6vh||||false|false” border_color_left=”rgba(0,0,0,0)”][et_pb_column type=”4_4″ _builder_version=”4.4.1″][et_pb_text _builder_version=”4.7.7″ text_font=”||||||||” text_font_size=”1.1em” text_line_height=”1.6em” quote_font=”|700|||||||” quote_text_align=”left” quote_font_size=”16px” header_2_text_color=”#ff6317″ header_2_font_size=”1.5em” header_2_line_height=”0.9em” header_3_font_size=”23px” header_4_font=”||||||||” header_4_font_size=”16px” header_4_line_height=”1.5em” header_5_font_size=”14px” custom_margin=”||||false|false” custom_padding=”||||false|false” border_color_left=”#ff6317″]<\/p>\n
In order to select a DNS service, we used that moment\u2019s status quo to develop the requirements:<\/p>\n
Based on these, we went with the Premium DNS package of ClouDNS: it requires our developers to bookmark a link in their regular browser, then click it\/open it to update their respective FQDN entry.<\/p>\n
Once the FQDN entry with info on the developer\u2019s updated IP was available, we needed to update the firewall information as soon as the IP address changed. By analysing our regular tools and technologies, we got to combine three of them as follows:<\/p>\n
So, let\u2019s look into each one:[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row use_custom_gutter=”on” _builder_version=”4.7.7″ custom_margin=”||||false|false” custom_padding=”6vh||0px||false|false” border_color_left=”rgba(0,0,0,0)”][et_pb_column type=”4_4″ _builder_version=”4.4.1″][et_pb_text _builder_version=”4.7.7″ text_font=”||||||||” text_font_size=”1.1em” text_line_height=”1.6em” quote_font=”|700|||||||” quote_text_align=”left” quote_font_size=”16px” header_2_text_color=”#ff6317″ header_2_font_size=”1.5em” header_2_line_height=”0.9em” header_3_font_size=”23px” header_4_font=”||||||||” header_4_font_size=”16px” header_4_line_height=”1.5em” header_5_font_size=”14px” custom_margin=”||||false|false” custom_padding=”||||false|false” border_color_left=”#ff6317″]<\/p>\n
We decided to use Python to solve the DNS part of the challenge because:<\/p>\n
Our script has three main goals:<\/p>\n
Here\u2019s the script extract:[\/et_pb_text][et_pb_text _builder_version=”4.7.7″ text_font=”|300|||||||” text_font_size=”1em” text_line_height=”1.3em” quote_font=”|700|||||||” quote_text_align=”left” quote_font_size=”1em” header_2_text_color=”#ff6317″ header_2_font_size=”1.5em” header_2_line_height=”0.9em” header_3_font_size=”23px” header_4_font=”||||||||” header_4_font_size=”16px” header_4_line_height=”1.5em” header_5_font_size=”14px” background_color=”#f8f8f8″ custom_margin=”|5vw||5vw|false|false” custom_padding=”||||false|false” border_width_left=”1px” border_color_left=”#bbbbbb” locked=”off”]<\/p>\n
#!\/usr\/bin\/env python3<\/span>\nimport<\/span> socket<\/span>\nimport<\/span> sys<\/span>\nimport<\/span> os<\/span>\nipfile =<\/span> 'ci\/dynip\/oldips'<\/span>\nhostnames =<\/span> [<\/span>\"DeveloperFirstName.DeveloperLastName.dyn.myDomain.com\"<\/span>]<\/span>\nips=<\/span>\"\"<\/span>\nold_ips=<\/span>\"\"<\/span>\ni=<\/span>1<\/span>\nfor<\/span> hostname in<\/span> hostnames:\n print<\/span>(<\/span>hostname)<\/span>\n ip =<\/span> socket<\/span>.gethostbyname<\/span>(<\/span>hostname)<\/span>\n ips=<\/span>ips+'{ip = \"'<\/span>+ip+'\/32\",name = \"'<\/span>+hostname+'\"}'<\/span>\n if<\/span> i !=<\/span> len<\/span>(<\/span>hostnames)<\/span>:\n ips=<\/span>ips+\",\\n<\/span>\"<\/span>\n i=<\/span>i+1<\/span>\nprint<\/span>(<\/span>'Discovered ips:'<\/span>)<\/span>\nprint<\/span>(<\/span>ips)<\/span>\n### Here tyo add check if the string ips is different to the one we have in the status file. <\/span>\n### If different then continue - else exit<\/span>\ndef<\/span> inplace_change(<\/span>filename_tmpl,<\/span> filename_dest,<\/span> old_string,<\/span> new_string)<\/span>:\n # Safely read the input filename using 'with'<\/span>\n with<\/span> open<\/span>(<\/span>filename_tmpl)<\/span> as<\/span> fr:\n s =<\/span> fr.read<\/span>(<\/span>)<\/span>\n if<\/span> old_string not<\/span> in<\/span> s:\n print<\/span>(<\/span>'\"{old_string}\" not found in {filename}.'<\/span>.format<\/span>(<\/span>**locals<\/span>(<\/span>)<\/span>)<\/span>)<\/span>\n return<\/span>\n\u00a0\n # Safely write the changed content, if found in the file<\/span>\n with<\/span> open<\/span>(<\/span>filename_dest,<\/span> 'w'<\/span>)<\/span> as<\/span> fw:\n print<\/span>(<\/span>'Changing \"{old_string}\" to \"{new_string}\" in {filename_tmpl}'<\/span>.format<\/span>(<\/span>**locals<\/span>(<\/span>)<\/span>)<\/span>)<\/span>\n s =<\/span> s.replace<\/span>(<\/span>old_string,<\/span> new_string)<\/span>\n fw.write<\/span>(<\/span>s)<\/span>\n## inplace_change('ci\/dynip\/secgroups-dynip-tmpl','secgroups-dynip.tf','!!!ADDRESSES!!!',ips)<\/span>\ndef<\/span> read_file(<\/span>filePathOldIps)<\/span>:\n\twith<\/span> open<\/span>(<\/span>filePathOldIps)<\/span> as<\/span> fr:\n\t\ts =<\/span> fr.read<\/span>(<\/span>)<\/span>\n\t\tprint<\/span>(<\/span>'Old ips:'<\/span>)<\/span>\n\t\tprint<\/span>(<\/span>'\"{s}\" found in \"{filePathOldIps}\".'<\/span>.format<\/span>(<\/span>**locals<\/span>(<\/span>)<\/span>)<\/span>)<\/span>\n\t\treturn<\/span> s\ndef<\/span> write_file(<\/span>filePathOldIps,<\/span> newIps)<\/span>:\n\twith<\/span> open<\/span>(<\/span>filePathOldIps,<\/span> 'w+'<\/span>)<\/span> as<\/span> fw:\n\t\tfw.write<\/span>(<\/span>newIps)<\/span>\n\t\tprint<\/span>(<\/span>'ips written in {filename}.'<\/span>)<\/span>\nif<\/span> os<\/span>.path<\/span>.isfile<\/span>(<\/span>ipfile)<\/span>:\n\told_ips =<\/span> read_file(<\/span>ipfile)<\/span>\nif<\/span> old_ips ==<\/span> ips:\n\tprint<\/span>(<\/span>'no ip was changed'<\/span>)<\/span>\n\tsys<\/span>.exit<\/span>(<\/span>666<\/span>)<\/span>\nelse<\/span>:\n\tprint<\/span>(<\/span>'ips were changed'<\/span>)<\/span>\n\twrite_file(<\/span>ipfile,<\/span>ips)<\/span>\n\tinplace_change(<\/span>'ci\/dynip\/secgroups-dynip-tmpl'<\/span>,<\/span>'secgroups-dynip.tf'<\/span>,<\/span>'!!!ADDRESSES!!!'<\/span>,<\/span>ips)<\/span><\/pre>\n[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row use_custom_gutter=”on” _builder_version=”4.7.7″ custom_margin=”||||false|false” custom_padding=”6vh||0px||false|false” border_color_left=”rgba(0,0,0,0)”][et_pb_column type=”4_4″ _builder_version=”4.4.1″][et_pb_text _builder_version=”4.7.7″ text_font=”||||||||” text_font_size=”1.1em” text_line_height=”1.6em” quote_font=”|700|||||||” quote_text_align=”left” quote_font_size=”16px” header_2_text_color=”#ff6317″ header_2_font_size=”1.5em” header_2_line_height=”0.9em” header_3_font_size=”23px” header_4_font=”||||||||” header_4_font_size=”16px” header_4_line_height=”1.5em” header_5_font_size=”14px” custom_margin=”||||false|false” custom_padding=”||||false|false” border_color_left=”#ff6317″]<\/p>\n
Terraform<\/h2>\n
To help with the maintenance and change management of our cloud infrastructure, we have been using infrastructure as a code solution from Terraform with the OpenTelekomCloud modules. (This is only because we are using the OTC cloud. The same result can be achieved by AWS \/ Azure with their respective modules.)<\/p>\n
Here we got to the next challenges. As the script was run automatically, we needed the possibility to automatically plan, approve and apply the changes only to the specific security group. All other changes would be ignored (e.g., any changes that could be caused by a new base image, or any incompletely checked change committed in the repo).<\/p>\n
Terraform is offering an elegant solution to this challenge, as it allows us to only execute a specific terraform file and to automatically approve the application of changes.<\/p>\n
This can be achieved using:[\/et_pb_text][et_pb_text _builder_version=”4.7.7″ text_font=”|300|||||||” text_font_size=”1em” text_line_height=”1.3em” quote_font=”|700|||||||” quote_text_align=”left” quote_font_size=”1em” header_2_text_color=”#ff6317″ header_2_font_size=”1.5em” header_2_line_height=”0.9em” header_3_font_size=”23px” header_4_font=”||||||||” header_4_font_size=”16px” header_4_line_height=”1.5em” header_5_font_size=”14px” background_color=”#f8f8f8″ custom_margin=”|5vw||5vw|false|false” custom_padding=”||||false|false” border_width_left=”1px” border_color_left=”#bbbbbb” locked=”off”]<\/p>\n
.\/ci\/app\/terraform plan -target=opentelekomcloud_compute_secgroup_v2.secgrp_everithyng_dynip -out=project-plan02.dat |& tee -a .\/logs\/plan-dyn-$today.log\n\n.\/ci\/app\/terraform apply -var-file=\".\/ci\/secrets\/secret.tfvars\" -auto-approve -target=opentelekomcloud_compute_secgroup_v2.secgrp_everithyng_dynip |& tee -a .\/logs\/planautopply-dyn-$today.log<\/pre>\n[\/et_pb_text][et_pb_text _builder_version=”4.7.7″ text_font=”||||||||” text_font_size=”1.1em” text_line_height=”1.6em” quote_font=”|700|||||||” quote_text_align=”left” quote_font_size=”16px” header_2_text_color=”#ff6317″ header_2_font_size=”1.5em” header_2_line_height=”0.9em” header_3_font_size=”23px” header_4_font=”||||||||” header_4_font_size=”16px” header_4_line_height=”1.5em” header_5_font_size=”14px” custom_margin=”||||false|false” custom_padding=”||||false|false” border_color_left=”#ff6317″]After the terraform changes are applied to the infrastructure, we also have to make sure the terraform.tfstate is automatically committed to the git. This way, next time when terraform gets to execute, it will be aware of the correct status.[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row use_custom_gutter=”on” _builder_version=”4.7.7″ custom_margin=”||||false|false” custom_padding=”6vh||0px||false|false” border_color_left=”rgba(0,0,0,0)”][et_pb_column type=”4_4″ _builder_version=”4.4.1″][et_pb_text _builder_version=”4.7.7″ text_font=”||||||||” text_font_size=”1.1em” text_line_height=”1.6em” quote_font=”|700|||||||” quote_text_align=”left” quote_font_size=”16px” header_2_text_color=”#ff6317″ header_2_font_size=”1.5em” header_2_line_height=”0.9em” header_3_font_size=”23px” header_4_font=”||||||||” header_4_font_size=”16px” header_4_line_height=”1.5em” header_5_font_size=”14px” custom_margin=”||||false|false” custom_padding=”||||false|false” border_color_left=”#ff6317″]<\/p>\n
GitLab CI\/CD<\/h2>\n
We use GitLab.com for source control and to run the CI\/CD pipelines from our project. We, therefore, had to enhance the gitlabci<\/em> script to add a new step that can be executed based on a condition. This step should be the only one that\u2019s executed when we start the pipeline from the scheduler.<\/p>\nStep details:[\/et_pb_text][et_pb_text _builder_version=”4.7.7″ text_font=”|300|||||||” text_font_size=”1em” text_line_height=”1.3em” quote_font=”|700|||||||” quote_text_align=”left” quote_font_size=”1em” header_2_text_color=”#ff6317″ header_2_font_size=”1.5em” header_2_line_height=”0.9em” header_3_font_size=”23px” header_4_font=”||||||||” header_4_font_size=”16px” header_4_line_height=”1.5em” header_5_font_size=”14px” background_color=”#f8f8f8″ custom_margin=”|5vw||5vw|false|false” custom_padding=”||||false|false” border_width_left=”1px” border_color_left=”#bbbbbb” locked=”off”]<\/p>\n
dynip-otc:\n stage: dynip\n before_script:\n - apk --no-cache add bash git openssl py-pip\n script:\n - .\/ci\/bin\/x_dynsecurity.sh\n tags:\n - project\n artifacts:\n paths:\n - .terraform\/\n - project-plan02.dat\n - logs\/\n expire_in: 1 week\n only:\n refs:\n - master\n variables:\n - $SCHEDULED_DYNIP =~ \/^true*\/<\/pre>\n[\/et_pb_text][et_pb_text _builder_version=”4.7.7″ text_font=”||||||||” text_font_size=”1.1em” text_line_height=”1.6em” quote_font=”|700|||||||” quote_text_align=”left” quote_font_size=”16px” header_2_text_color=”#ff6317″ header_2_font_size=”1.5em” header_2_line_height=”0.9em” header_3_font_size=”23px” header_4_font=”||||||||” header_4_font_size=”16px” header_4_line_height=”1.5em” header_5_font_size=”14px” custom_margin=”||||false|false” custom_padding=”||||false|false” border_color_left=”#ff6317″]We also needed to create a new scheduler in GitLab to execute this step. This can be done from the GitLab interface:[\/et_pb_text][et_pb_image src=”https:\/\/berg-software.com\/wp-content\/uploads\/DynamicIP-remote-access-GitLab-01.jpg” alt=”DynamicIP remote access GitLab 01″ title_text=”DynamicIP remote access GitLab 01″ _builder_version=”4.7.7″ max_width=”60vw” custom_padding=”2vh||2vh|2vw|false|false” locked=”off”][\/et_pb_image][et_pb_text _builder_version=”4.7.7″ text_font=”||||||||” text_font_size=”1.1em” text_line_height=”1.6em” quote_font=”|700|||||||” quote_text_align=”left” quote_font_size=”16px” header_2_text_color=”#ff6317″ header_2_font_size=”1.5em” header_2_line_height=”0.9em” header_3_font_size=”23px” header_4_font=”||||||||” header_4_font_size=”16px” header_4_line_height=”1.5em” header_5_font_size=”14px” custom_margin=”||||false|false” custom_padding=”||||false|false” border_color_left=”#ff6317″]Finally, we needed to add a new schedule with the following details:[\/et_pb_text][et_pb_image src=”https:\/\/berg-software.com\/wp-content\/uploads\/DynamicIP-remote-access-GitLab-02.jpg” alt=”DynamicIP remote access GitLab 02″ title_text=”DynamicIP remote access GitLab 02″ _builder_version=”4.7.7″ max_width=”60vw” custom_padding=”2vh||2vh|2vw|false|false” locked=”off”][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row use_custom_gutter=”on” _builder_version=”4.7.7″ custom_margin=”||||false|false” custom_padding=”6vh||0px||false|false” border_color_left=”rgba(0,0,0,0)”][et_pb_column type=”4_4″ _builder_version=”4.4.1″][et_pb_text _builder_version=”4.7.7″ text_font=”||||||||” text_font_size=”1.1em” text_line_height=”1.6em” quote_font=”|700|||||||” quote_text_align=”left” quote_font_size=”16px” header_2_text_color=”#ff6317″ header_2_font_size=”1.5em” header_2_line_height=”0.9em” header_3_font_size=”23px” header_4_font=”||||||||” header_4_font_size=”16px” header_4_line_height=”1.5em” header_5_font_size=”14px” custom_margin=”||||false|false” custom_padding=”||||false|false” border_color_left=”#ff6317″]<\/p>\n
Conclusion<\/h2>\n
By using no more than three simple and accessible tools (Python, Terraform, GitLab), we were able to:<\/p>\n