GET

PowerShell Sftp Retrieve multiple Files (and Remove Source files)

/ PowerShell Scripting /

The PowerShell script below retrieves multiple files from a SFTP server source using the native windows SFTP-client in windows After retrieving each individual file it removes the file from the SFTP-server. This specific script makes us of a (preconfigured) ssh-key authentication configuration, but it can easily been rewritten for password authentication.

<#
    Description    : Retrieve (get) and remove files from a sftp source... 
    Keywords       : Sftp, Get and Remove Original, Move

    Majorversion   : 1
    Scriptserver   : LOCALHOST
    Serviceaccount : -

    Author         : Tim van Kooten Niekerk
    Date           : 2023-03-14
#>

<#      CHANGELOG:
        2023-03-14 => Initieel (Tim van Kooten Niekerk)

#>


function fnMoveSftpFiles()
{
  param(
    [string]$sSftpServerHost,
    [string]$sSftpUser,
    [string]$sSftpServerPort = "22",
    [string]$sSftpRemoteFilePath,
    [string]$sSftpRemoteFileMask,
    [string]$sSftpLocalFilePath,
    [string]$sSftpScriptRetrieveClose = "quit"
  )

  # System variables...
  [string]$_sSftpScriptLogFile = $sSftpLocalFilePath + "\get-SftpFiles_" + $(get-date -f yyyy-MM-dd) + '.log'
  [string]$_sSftpScriptRetrieveFiles = "lcd " + $sSftpLocalFilePath + "`ncd " + $sSftpRemoteFilePath + "`n" + "ls -1 " + [string]$sSftpRemoteFileMask
  [string]$_sSftpScriptRetrieveOpen = "lcd " + $sSftpLocalFilePath + "`ncd " + $sSftpRemoteFilePath + "`n"
  [string]$_sSftpScriptRetrieveBody = $_sSftpScriptRetrieveOpen
  [string]$_CurrentTimeStamp = $(get-date -f 'yyyy-MM-dd HH:mm:ss')
  [string]$_InfoDoneWithFiles = $_CurrentTimeStamp + " | INFO | Commands executed: "
  [string]$_InfoDoneWithoutFiles = $_CurrentTimeStamp + " | INFO | No remote files found"
  [string]$_Return = $_CurrentTimeStamp + " | ERROR | Unknown Error"

  # Retrieve file list...
  [array]$aFiles = (write-output $_sSftpScriptRetrieveFiles) | sftp -b - -o Port=$sSftpServerPort $sSftpUser@$sSftpServerHost | select-string -notmatch "sftp>"

  # Create script for retrieving files...
  if ($aFiles.length -gt 0) {
    # Create get and rm script lines for every file in the listing...
    for (($n=0); $n -lt $aFiles.length; $n++) { 
      if ($aFiles[$n].line.length -gt 0)
      { 
        $_sSftpScriptRetrieveBody = $_sSftpScriptRetrieveBody + "get " + $aFiles[$n].Line + "`n"
        $_sSftpScriptRetrieveBody = $_sSftpScriptRetrieveBody + "rm " + $aFiles[$n].Line + "`n"
      }
    }

    # Close script file...
    $_sSftpScriptRetrieveBody = $_sSftpScriptRetrieveBody + $sSftpScriptRetrieveClose

    # Process script...
    [array]$aProcess = (write-output $_sSftpScriptRetrieveBody) | sftp -b - -o Port=$sSftpServerPort $sSftpUser@$sSftpServerHost
  
    # Audit log - Commands executed...
    $_InfoDoneWithFiles + $aProcess | Out-File $_sSftpScriptLogFile -Append
    $_Return = $_InfoDoneWithFiles + $aProcess
    [array]$aProcess = ""
  
  } else {
    # Audit log - No remote files...
    $_InfoDoneWithoutFiles | Out-File $_sSftpScriptLogFile -Append
    $_Return = $_InfoDoneWithoutFiles

  }
  return $_Return
}

PS> fnMoveSftpFiles -sSftpServerHost “example.com” -sSftpUser “user” -sSftpRemoteFilePath “/test” -sSftpRemoteFileMask “*.txt” -sSftpLocalFilePath “D:\Tim\Test” [-sSftpServerPort “22“]

PowerShell Retrieve OAUTH Access Token

/ PowerShell Scripting /

PowerShell function example to retrieve an OAUTH2 access token from ADFS using form-based authentication or by using a previously retrieved OAUTH Refresh Token.

function fnGetOauthXSToken()
{
  param (
    [string]$ADFSOAUTHAuthorizeUri, 
	[string]$ADFSOAUTHGetTokenUri, 
	[string]$Client_ID, 
	[string]$Resource, 
	[string]$Redirect_URI,
	[string]$RefreshToken, 
	[string]$UserName, 
	[string]$Password,
	[string]$Secret
	)

  if ( $RefreshToken -ne "" ) { 
    
    # Get OAUTH Access Token by using OAUTH refresh_token...

    ## Retrieve OAUTH Token...
    $vPostValues = "grant_type=refresh_token&client_id=" + $Client_ID + "&redirect_uri=" + $Redirect_URI + "&refresh_token=" + $RefreshToken
    $oResult0 = Invoke-RestMethod -Method Post -Uri $ADFSOAUTHGetTokenUri -UserAgent "Windows-AzureAD-Authentication-Provider" -Body $vPostValues -ContentType application/x-www-form-urlencoded

  } else {
    
    # Get OAUTH Access Token by using authorization_code (username and password)... 
  
    ## Build authentication Uri and create websession...
    $sUri = $ADFSOAUTHAuthorizeUri + "?response_type=code&client_id=" + $Client_ID + "&resource=" + $Resource + "&redirect_uri=" + $Redirect_URI
    $oWebSession = New-Object Microsoft.PowerShell.Commands.WebRequestSession

    ## Authenticate by using username and password (formbased)...
    $aPostValues = @{UserName=$UserName; Password=$Password; AuthMethod='FormsAuthentication' }
    $oResult0 = Invoke-WebRequest -Method Post -Uri $sUri -UserAgent "Windows-AzureAD-Authentication-Provider" -Body $aPostValues -Websession $oWebSession -MaximumRedirection 0 -ErrorAction SilentlyContinue

    ## Retrieve authorization code...
    $oResult0 = Invoke-WebRequest -Uri $sUri -UserAgent "Windows-AzureAD-Authentication-Provider" -Websession $oWebSession -MaximumRedirection 0 -ErrorAction SilentlyContinue
    $sCode = $oResult0.Headers.Location.Substring($oResult0.Headers.Location.IndexOf("?code=") + 6, ($oResult0.Headers.Location.Length - ($oResult0.Headers.Location.IndexOf("?code=") + 6) ) )
  
    ## Cleanup websession...
    $oResult0 = $null
    $oWebSession = $null
    $aPostValues = $null
  
    ## Retrieve OAUTH Token...
    if ($Secret -eq "") {
      $vPostValues = "grant_type=authorization_code&client_id=" + $Client_ID + "&redirect_uri=" + $Redirect_URI + "&code=" + $sCode
    } else {
      $vPostValues = "grant_type=authorization_code&client_id=" + $Client_ID + "&redirect_uri=" + $Redirect_URI + "&code=" + $sCode + "&client_secret=" + $Secret
    }
    $oResult0 = Invoke-RestMethod -Method Post -Uri $ADFSOAUTHGetTokenUri -UserAgent "Windows-AzureAD-Authentication-Provider" -Body $vPostValues -ContentType application/x-www-form-urlencoded
  
  }
  
  # Return Result and Cleanup...
  return $oResult0
  $vPostValues = $null
  $oResult0 = $null
}

PowerShell TCP Socket Function

/ PowerShell Scripting /

The PowerShell function example below can be used to transmit information using the TCP protocol. For example, to perform an HTTP GET command on a website.

function fnTCPSock([string]$sHostName, [string]$sPort, [string]$sScript) {
  # Converteer script string to byte for write stream...
  [byte[]]$byteScript = [System.Text.Encoding]::ASCII.GetBytes($($sScript))
  # Open 4096 Bytes buffer for read stream...
  [byte[]]$byteBuffer = New-Object System.Byte[] 16384
  
  # Connect to host...
  $oTCPSock = New-Object System.Net.Sockets.TcpClient($sHostName, $sPort)
  $oStream = $oTCPSock.GetStream() 
  
  # Feed script(byte) tot write stream...
  $oStream.Write($byteScript, 0, $byteScript.Length)

  # Return read buffer converted to string...
  [string]$sCount = $oStream.Read($byteBuffer, 0, 16384)
  return [System.Text.Encoding]::ASCII.GetString($byteBuffer, 0, $sCount) 

  # Close objects...
  $oResult.Close()
  $oSockTCP.Close()
}
[string]$sScript1 = "GET / HTTP/1.1`r`nAccept: text/html`r`nHost: [virtualhostname]`r`nReferer: http://host.domain.name/site/`r`n`r`n`r`n"
[string]$sResult1 = fnTCPSock "hostname" "80" $($sScript1)