1.5M ratings
277k ratings

See, that’s what the app is perfect for.

Sounds perfect Wahhhh, I don’t wanna

I figured out how to get ESRGAN (and SFTGAN) running on Windows. Unfortunately, it’s not as user friendly as waifu2x, but it’s fairly easy as far as Python deep learning programs go.

The most up to date info is at Reddit’s /r/gameupscale. https://www.reddit.com/r/GameUpscale/ You can also check out the Game Upscale Discord here: https://discord.gg/jkFHzGD 

I’m leaving this guide up because it still gets quite a bit of traffic.


You will need:

Windows 7 or up, 64-bit only

An Nvidia GPU is recommended. the newer and the more memory, the better. My 6GB 1060 could upscale 1024x600 images to 4096x2400. You’ll need to run it in CPU mode otherwise - see step 3.5.

Python 3 64-bit (x86-64) : https://www.python.org/downloads/release/python-366/

Step 1: Install CUDA 9.2 or 10.0, plus the patch if there is one. I believe you only need the CUDA and driver components: https://developer.nvidia.com/cuda-toolkit-archive

Step 2: Go to the PyTorch installation site. Use “pip” as the package, “stable” for the build, and everything else should match your software versions. Open up the Windows command prompt (search “cmd” in the start menu) and paste in the commands. https://pytorch.org/get-started/locally/

If it tells you “x is not a supported wheel on this platform”, make absolutely sure your options match and that your Python installation is 64-bit, or try different versions of Python or CUDA.

While you’re there, paste in this (pip3, not pip): pip3 install numpy opencv-python

Step 3: Download https://github.com/xinntao/ESRGAN along with the two pretrained models.

Step 3.5 (READ IF YOU HAVE AN AMD/INTEL GPU): Go into test.py and change “device = torch.device(‘cuda’)“ to “ device = torch.device(‘cpu’)”.

Step 4: Follow the instructions in ESRGAN’s “Quick Test” section. I highly recommend running the network interpolation command, because the default, full monty SRGAN model can make images a bit noisy whereas PSNR is too blurry. I usually use 0.8.

Ignore any “nn.Upsampling is deprecated” errors, they have no effect on the results. Do not use PNGs with alpha channels or you will get an “index out of range” error.k

Make sure to include the model path, or else you will get an “IndexError: list index out of range“ error. Either navigate to the ESRGAN directory via the command prompt, or save the commands you want as a .bat file in the ESRGAN directory.


.

Optional Step 5, for SFTGAN: Download SFTGAN here and follow the PyTorch instructions under “Test Models”. Make sure to download the pretrained models: https://github.com/xinntao/SFTGAN

SFTGAN is similar to ESRGAN. It tries to make more intelligent guesses about what kind of details it’s trying to retrieve, and it looks better in some cases (outdoor photographs, textures) and worse in others. The requirements are the same as ESRGAN.

Just Cause 2 texture comparison: https://imgur.com/a/JDsvVei

Place the images for  SFTGAN in data/samples. Unlike ESRGAN, you need to scale up the images by 4x before you run it. SFTGAN will downscale the image, then upscale it again, leaving the resolution unchanged. I use nearest neighbor scaling, but cubic scaling could be worth a try if dithering’s a problem.


Additional Info: I’ve been seeing this tutorial on other sites now and there’s a couple of things I need to clear up:

1. ESRGAN’s default networks were trained on a very specific type of image. That is, high resolution photographs downscaled with bicubic sampling. So it struggles with pics that went through color reduction or heavy dithering (like you see with old prerendered sprites), pictures that are blurry (I’d recommend downscaling them by 50%, as you’ll still get a 2x resolution boost), and pixel art. It may be possible to “tune” it by training it on specific categories of images. I’ve had some success adding small amounts of noise to the images to help it out, or blurring out color banding or sharp edges.

Other tips here:  https://boards.fireden.net/v/thread/443447437/#443455703

2. ESRGAN and SFTGAN are not the only super-resolution programs out there. The former is a modification of the existing SRGAN program from 2 years ago, and there’s new ones coming out every week. Not all of them exist as open source programs and not all of them run on Windows (Ubuntu 16.04 seems to be the “standard” OS for neural network programs), but I recommend you give them a try too, since there’s no one best way of upscaling photos. You could even shell out cash for Let’s Enhance or AI Gigapixel if you have more money than time.

3. Waifu2x is (as far as I know) the state of the art for illustrations and it’s what I’d recommend for HD anime reaction images.

4. NVIDIA is working on their own superresolution program. Right now it’s in closed beta (I’ll provide a link if they accept my application), but once it’s available to the public it’ll likely be the standard for super-resolution, and you’ll start seeing game developers use it themselves. https://developer.nvidia.com/gwmt

esrgan neural networks superresolution super-resolution neural network machine learning prosr ESR

More you might like

Anonymous asked:

Hi, i've been following your steps for the ESRGAN method. I'm stuck at step 2 really...i installed python 3.7.# (64bit), then i installed cuda 10. When i get to the pytorch site, i select stable, pip, python 3.7, cuda 10, and i get 2 commands. i open up cmd : paste that first command and i get : 'pip3' is not recognized as an internal or external command,operable program or batch file. So i'm not sure what's happening.

I’m not sure exactly, but maybe typing “pip” over “pip3″ would work.

(Also, this blog is inactive. Try asking /r/gameupscale)

ESRGAN training on Windows instructions

This guide is outdated. Please check https://www.reddit.com/r/GameUpscale/ for more up-to-date info.

Note: ESRGAN training appears to be slower on Windows than Linux by around 5x, at least on my machine. I don’t know the cause of this.

If you haven’t gotten ESRGAN set up for testing, please read this. Everything needed to test ESRGAN is also needed to train it - Python, CUDA, etc. https://kingdomakrillic.tumblr.com/post/178254875891/i-figured-out-how-to-get-esrgan-and-sftgan

If you’ve already done all that, go to Step 1.

1: Download and install Microsoft Build Tools 2015. It’s needed for one of BasicSR’s dependencies.

Then go to the command line and paste in this: pip install numpy opencv-python lmdb

2. Download BasicSR and the ESRGAN pretrained models.

https://github.com/xinntao/BasicSR
https://github.com/xinntao/BasicSR#pretrained-models

Place the models in (BasicSR directory)/experiments/pretrained_models

3. Download a dataset. The BasicSR creator uploaded several datasets to use here, but there’s plenty of other datasets you can use online. 1000 tiles (see step 5) are the absolute minimum for getting good results, but the more, the better.

https://github.com/xinntao/BasicSR#datasets

Make absolutely sure that none of the images are greyscale or indexed, or have alpha channels. RGB only,, or else you will get a “Sizes of tensors must match” error. You can use InfranView or BIMP (see below) to convert the images to RGB.

4 You will need to split your “training” and “validation” images. Take about 5-10% of your images and put them  in a separate folder; these will be your validation images.

5. You will also need to convert your dataset into fixed tiles. Open up codes/scripts/extract_subimgs_single.py.

Change crop_sz to 192 or 128 (I’d stick to the latter unless you have a beefy graphics card), input_folder to the full path name of your image folder, and save_folder to where you want to save the tiles to. If you’re using Windows, replace all the slashes (“") with double slashes, as ”" is an escape character.

Example:

input_folder = ‘C:\Users\Username\BasicSR-master\General100’

save_folder = 'C:\Users\Username\BasicSR-master\General100_tiles’

Double click to run it. Repeat this process for the validation images.

6. You will need to batch convert these HR tiles to 4x downscaled versions. Download and open InfranView (https://www.irfanview.com/),  press B to open the batch convert dialog, check “Use advanced options” and then click “Advanced” button to access the resize settings. You may want to check “Change Color Depth” or add some JPG compression, noise or dithering if you’re specifically training it for low quality images. It also helps make the results smoother overall. Make sure that both the LR and HR images have the same format and filename.

If you have GIMP installed, you can also download a batch manipulation plugin called BIMP and process the image that way.

7. Go to codes/options/train/train_ESRGAN.json and make the following changes:

name: change to whatever you want, removing the “debug” from the name

scale: Default is 4, meaning that the HR images are 4x larger than the LR images. I’d recommend leaving it as it is, but if you do set the value to something else, you will need to alter path: {pretrain_model_G (see below)

train : { dataroot_HR: location of the training HR images

train : { dataroot_LR: location of the training LR images

val : { dataroot_HR: location of the HR validation images

val : { dataroot_LR: location of the LR validation images

path : { root: the location of the BasicSR directory

train : { HR_size: the size of the HR tiles. Leave at 128 if you’re getting “out of memory” errors.

train : { batch_size: You could lower this number if you’re getting “out of memory” errors, but that produces other errors on my Windows installation. “n_workers” may be an alternative.

train : { val_freq: How often the model will be validated. Defaults to 5,000 iterations (5e3), so feel free to lower it.

path: {pretrain_model_G: The model ESRGAN will initialize from. If using a scale other than 4, set to “null” without quotes

logger: { print_freq: How often the program will update you on how many iterations have passed. Set this to as low as 1 if you’d like

logger: { save_checkpoint_freq: How often a new model will be saved. Set it to the same number as val_freq.

Again, make sure to use double shashes.

8. Use the command line to navigate to the codes folder and run this command: python train.py -opt options/train/train_esrgan.json

You could also create a .bat file so you can just double click, though that does make it harder to find errors (as an error will close the command prompt instantly).

9. You can check on the model’s progress by going into the “experiments” folder. Your older sessions will have an  "archived" in their name, while the latest session will not. Inside each folder is the “models” folder, which is where new models are saved, and the upscaled validation images will appear in “val_images”. Once you’re satisfied, hit Ctrl-C in the terminal to quit training and copy one of the “G.pth” files to ESRGAN’s models folder.

The “training_state” folder contains .state files that let you resume progress after you’ve stopped. Just add -resume_state next time you run train.py, plus the path to the file.

If you’re feeling brave, you can mess with the GAN weight, feature weight and pixel weight in train_ESRGAN.json or initialize from a different model instead of RRDB_PSNR_x4.pth

i had planned to leave tumblr but that esrgan post blew up still waiting for pillowfort to go public esrgan

Anonymous asked:

Hello, how can I include model path (ESRGAN)? Cause then I want to lauch something there is "IndexError: list index out of range".

python test.py models/RRDB_ESRGAN_x4.pth (or whatever model you’re using)

Either use the command prompt to navigate to the ESRGAN directory, or take that command, paste it into Notepad, and save it to the ESRGAN directory as a .bat file.

Manga109 model attempt (for illustrations): http://www.mediafire.com/file/w3jujtm752hvdj1/Manga109Attempt.pth.zip/file

List of models made by the /r/GameUpscale community:https://upscale.wiki/wiki/Model_Database

I’ve spent the night training ESRGAN on the Manga109 dataset. Compared to the default model, it’s more soft and more subtle, but it handles dithering well You can also interpolate it with the existing networks by modifying net_interp.py.

There’s a lot of room for improvement. If you want to train it yourself, read https://github.com/xinntao/BasicSR and https://github.com/xinntao/BasicSR/tree/master/codes/data

I used ImageMagick to turn the dataset into 128x128 tiles (any higher and it ran out of memory), then downscaled them to 32x32 for the low res dataset. I didn’t use BasicSR’s matlab downscaler nor did I make an LMDB file. Make absolutely sure that all tiles are the same size, that the HR and LR versions have the same filenames, and that they’re all RGB, no greyscale or indexed images (or else you’ll get a “Got 3 and 1 in dimension 1″ error). You’ll need to put the images in 4 folders: 80-90% of the pics into “training” folders and 10-20% into “validation” folders, and separate folders for HR and LR images. I also went into options.py and changed val_frequency and save_checkpoint_freq to 256 (too many checkpoints will eat up your hard drive space), and in train_esrgan.json I turned down the batch size and number of workers to keep memory usage down.

The experiments/ folder contains the validation results and the models (specifically the files ending in G.pth).

It’s a massive pain to set up, I haven’t tried training it on Windows and I probably left out vital info. I don’t even know if I’m training it “correctly” because I left the GAN settings untouched. Good luck!

The BasicSR creator uploaded some image datasets here: https://drive.google.com/drive/folders/1pRmhEmmY-tPF7uH8DuVthfHoApZWJ1QU

If I (or anyone else) create any more models, I’ll put them in the top of this post. But I think I’ve wrung all I can out of ESRGAN.

esrgan srgan neural network R

A neural network generates TV Tropes trope names

For those fortunate enough to not know, TV Tropes is a site that catalogues tropes, which are repeated conventions in media that serve as a narrative shorthand, such as plot points and character traits. All cliches are tropes, but not all tropes are cliches. Each are given their own “colorful” name, such as “New Powers As The Plot Demands” or “Locking MacGyver In The Store Cupboard“ or “A Tankard of Moose Urine”.

Conveniently, TV Tropes has a massive page listing all ~25,000 tropes on the site, from “Abandoned Hospital” to “Zipping Up The Body Bag”. I took all of that plus 3,000 from the old Trope Bin page, making 2 copies and randomly shuffling the lines so that trope names don’t get influenced by adjacent names in the list. All of this was run through torch-rnn.

In theory, the program’s temperature setting controls how “creative” the generated text is vs. how “stereotypical” it is, with 0.1 being as stereotypical as possible. In practice, torch-rnn has an odd bias towards words beginning with “St”. It’s been true every time I’ve run it and this is no exception:

Start of the Story
Star Transformation
Stock Shot
Star Shaped Car
Star Ship Santa
School Stars Are Different
The Star Of Stealing
Stock Shoujo Bullying Time

(note: none of the names I post here exist in the original dataset)

It mostly vanishes when I up it to 0.3, but it still has a fixation on “S” which manifests in a boatload of sex tropes.

Non Lethal Sex
Sexy Captain
Sex With A Horse
Magic Sex
Sex Montage
Self Destructive Beautiful Sex
Badass Background Sexual Man
Instant Death By Sexy
Shoot The Sexy Character
Final Sexuality
Super Sex Song
Sexy Carnival
Sex Is A Special Kind Of Lawyer
Sexy School Sex
Sex Changes Are Fish
Sexy Monkey To Scale
The Sexy Family
Anti Sex Angel
Dead Man Sex
Screw The Children

Some generated trope names were so realistic that I had to  make absolutely sure they weren’t in the original dataset:

The Show Must Be The Creators Life
Canon Hates You
Death By God
Bad Guys Are Burning
Promotion Magnet
Matchmaker Girl
And Then I Was A Parody
The Magnificent States Of America
Ethnic Superpower
Big Bad John
Love Is A Monster
Parody Swearing
Big Brother Is Not Evil
The Magnificent Stupid
Interspecies Apocalypse
All Gays Are Different
Illegal Species
Department Of Doom
Where I’m Not A Virgin

While others sound like tropes from an alternate universe:

Panty Tutorial
Jazz Fight
The Baby Of The Ages
Star Shipping
Trapped In The Moon
Scream Suit
Action Party Marriage
The Based For The God
Mistaken For Cat
Superpowered Doors
Never Bring A Bar Stop On The Mob
Beehive Heroism
Chekov’s Gender
Meddling Dick
I Can’t Believe It’s Not Anthologist
What The Hell Is A Cold

And inevitably, many are complete nonsense, especially when the temperature is set above 1:

Necroshiposhop
Blood Is Betty
Mistaken For An Ico
Jailbows!
Bith Ol Apa Bangspoof
Wolf Blood Staircase
Revolutionary Wound Bad Gay
Zaroh Butterfuff
What Owe Them Thing Her To Say There) Disney’s Kick
Flupy Eye Clown
Black Scientists In My Smade
Wolfornia Is Hands,men
I Lub, Ghes’t Have TheyCKild Me
OM NKH’ No! Will Rose With Angst)
HHHHHHHHEL!
“MOOG 62C:- OONWa GaOp
(the number 7 repeated 50,000 times)
a markov chain would have worked just as well but somehow it doesn't seem as cool neural network neural networks tv tropes tvtropes i hope janelle shane wasn't planning on doing this
See this in the app Show more