Remove Powershell History Duplicate Commands

You can avoid displaying duplicate commands in powershell with this simple command:

Code:

Set-PSReadLineOption –HistoryNoDuplicates:$True

But even so, your ConsoleHost_history.txt gets filled with duplicate lines, and after a while it gets tedious to find and copy/past commands from that file.

Here's a simple script that will remove duplicate lines from PSReadline history, and off course leave them in the same order.

Code:

@echo off
%windir%\system32\reg.exe query "HKU\S-1-5-19" 1>NUL 2>NUL || goto :NOADMIN

::Set Variables
setlocal disableDelayedExpansion
set "file=%AppData%\Microsoft\Windows\PowerShell\PSReadLine\ConsoleHost_history.txt"
set "line=%file%.line"
set "deduped=%file%.deduped"
::Define a variable containing a linefeed character
set LF=^


::The 2 blank lines above are critical, do not remove
>"%deduped%" (
  for /f usebackq^ eol^=^%LF%%LF%^ delims^= %%A in ("%file%") do (
    set "ln=%%A"
    setlocal enableDelayedExpansion
    >"%line%" (echo !ln:\=\\!)
    >nul findstr /xlg:"%line%" "%deduped%" || (echo !ln!)
    endlocal
  )
)
>nul move /y "%deduped%" "%file%"
2>nul del "%line%"
exit /b

:NOADMIN
echo You must have administrator rights to run this script.
<nul set /p dummyName=Press any key to exit...
pause >nul
goto :eof

I was doing that task since long time with Notepad++ search and replace function

Code:

^(.*?)$\s+?^(?=.*^\1$)

Replace with nothing, check "Regular expression" and ". matches newline".
I even saved this as a macro and made a vbs script to run that macro

But today (hooray!) I finally found a way to do it with a simple batch script...
The only difference is that .bat script will keep 1st "deduplicated command" (highest), while Notepad++ will keep last (lowest) deduplicated command.

Here are the Notepad++ files I made if anyone interested, who knows...
Might be also interesting for people who search how to launch Notepad++ macros from a script (I searched myself for quite some time)

Here's the customized shortcut.xml file (with the saved macro):
path: %AppData%\Notepad++\shortcut.xml

Code:

<NotepadPlus>
    <InternalCommands />
    <Macros>
        <Macro name="DeleteDuplicateLines" Ctrl="yes" Alt="no" Shift="yes" Key="68">
            <Action type="3" message="1700" wParam="0" lParam="0" sParam="" />
            <Action type="3" message="1601" wParam="0" lParam="0" sParam="^(.*?)$\s+?^(?=.*^\1$)" />
            <Action type="3" message="1625" wParam="0" lParam="2" sParam="" />
            <Action type="3" message="1602" wParam="0" lParam="0" sParam="" />
            <Action type="3" message="1702" wParam="0" lParam="1794" sParam="" />
            <Action type="3" message="1701" wParam="0" lParam="1609" sParam="" />
            </Macro>
        <Macro name="Trim Trailing Space and Save" Ctrl="no" Alt="yes" Shift="yes" Key="83">
            <Action type="2" message="0" wParam="42024" lParam="0" sParam="" />
            <Action type="2" message="0" wParam="41006" lParam="0" sParam="" />
        </Macro>
    </Macros>
    <UserDefinedCommands>
       <Command name="Launch in Firefox" Ctrl="yes" Alt="yes" Shift="yes" Key="88">firefox &quot;$(FULL_CURRENT_PATH)&quot;</Command>
       <Command name="Launch in IE" Ctrl="yes" Alt="yes" Shift="yes" Key="73">iexplore &quot;$(FULL_CURRENT_PATH)&quot;</Command>
       <Command name="Launch in Chrome" Ctrl="yes" Alt="yes" Shift="yes" Key="82">chrome &quot;$(FULL_CURRENT_PATH)&quot;</Command>
       <Command name="Launch in Safari" Ctrl="yes" Alt="yes" Shift="yes" Key="65">safari &quot;$(FULL_CURRENT_PATH)&quot;</Command>
       <Command name="Get PHP help" Ctrl="no" Alt="yes" Shift="no" Key="112">http://www.php.net/$(CURRENT_WORD)</Command>
       <Command name="Wikipedia Search" Ctrl="no" Alt="yes" Shift="no" Key="114">https://en.wikipedia.org/wiki/Special:Search?search=$(CURRENT_WORD)</Command>
       <Command name="Open file in another instance" Ctrl="no" Alt="yes" Shift="no" Key="117">$(NPP_FULL_FILE_PATH) $(CURRENT_WORD) -nosession -multiInst</Command>
        <Command name="Send via Outlook" Ctrl="yes" Alt="yes" Shift="yes" Key="79">outlook /a &quot;$(FULL_CURRENT_PATH)&quot;</Command>
    </UserDefinedCommands>
    <PluginCommands />
    <ScintillaKeys />
</NotepadPlus>

And here's the vbs script to call (and run) that macro (and close Notepad++)
ClearPsReadlineDuplicates.vbs

Code:

Dim objShell
Set objShell = WScript.CreateObject("WScript.Shell")
objShell.run("notepad++.exe ""%AppData%\Microsoft\Windows\PowerShell\PSReadline\ConsoleHost_history.txt""")
WScript.Sleep(100)
objShell.SendKeys("^+D")
Dim objShell2
Set objShell2 = WScript.CreateObject("WScript.Shell")
objShell.run("notepad++.exe ""%AppData%\Microsoft\Windows\PowerShell\PSReadline\ConsoleHost_history.txt""")
WScript.Sleep(100)
objShell2.SendKeys("^+D")
objShell2.SendKeys("^S")
objShell2.SendKeys("^+W")
objShell2.SendKeys "% C"

 

I made a new one using the Regex I use in Notepad++ and with the help of JRepl.bat utility.
It will keep last duplicate line, which is much better for Powershell.
Most credits goes to dbenham for his awesome Jrepl script.

https://github.com/Thdub/Batch_Scripts/blob/master/RemovePowershellDuplicateLines.bat